Something just happened. Something big. The wheels are falling off the enterprise. People can't log in. "The XYZ domain is not available. Please try again later." Outlook is endlessly prompting folks for credentials. No one can print. Many can't even get an IP address. Mission-critical apps are erroring out left and right. The Helpdesk phone tree is lit up like a Christmas tree. People whose title starts with "C" are briskly walking the floor, asking questions. Desk and cell phones are ringing all over the place. Stomachs are turning. Throats are tightening up, especially for one person in particular (but who?).
The "meter is running" but no one is going anywhere….business is down.
After some craziness and high-stress troubleshooting, it turns out someone deleted the main, internal corporate DNS zone. An authoritative restore was done on the zone and things returned to normal but the outage lasted for several hours and $$$ revenue and productivity were lost. The reputation of IT took a hit but we're glad we had viable and tested AD backups and recovery plans (do you? If not, go here for some tips and get started!):
Now, people want answers. What happened? Who did it? When did it happen? Why did it happen?
Sound familiar? Luckily, this was a fictional situation - but based on true stories. If you've followed this blog or gone Internet searching for auditing in Windows, you've likely seen the posts here and elsewhere about auditing AD for OU, GPO and other AD deletions, file and folder edits and possibly failed user logons. DNS auditing is much less frequently covered. Even in my prior post, I didn't delve into DNS auditing.
We know how critical DNS is to AD, and how critical AD is to the enterprise. So let's get proactive and configure some settings to help ensure we are positioned to answer those "W" questions for DNS.
Once the word gets out that 'we're watching AD changes like a hawk,' we often see a drastic reduction in something I refer to as 'casual administration.' Casual administration is when well-intentioned admins start poking around MMC consoles, tinkering. No change-control requests or communications have gone out. This is just someone trying make things better without making any ripples, implementing seemingly minor changes. Or, perhaps it was an attempt to clean-up DNS, troubleshoot an AD replication error or someone following along on an Internet blog post without testing it in a lab first.
My co-conspirators in this endeavor, BrentW and Bryan Zink, thought to parlay our discussions around this critical proactive effort into a blog post to present an auditing solution for DNS data.
Before we start, you should review my prior post for a primer into AD auditing, some very important caveats and a bit more depth, then come right back…
Okay, picking up from there…
For AD-integrated DNS changes, we actually audit AD objects and recall that AD auditing is a two-step process:
Step 1 – Configure the desired/correct 'Audit Policy' on the DCs
Step 2 – Turn on the appropriate auditing settings on the target object(s)
Step 1: The configuration for the Audit Policy on the Domain Controllers for this effort is shown in the screenshots below:
Step 2 – Configure auditing on the target object(s).
See the screenshots below for the audit settings for zone creation:
NOTE: Due to the way DNS deletions are processed in AD, if we enable auditing for "Delete dnsZone objects" here, we won't see Zone deletions in the logs. We'll cover deletions in a minute.
See the bottom of this post for the unique details about DNS deletions from AD-integrated DNS.
After we setup the auditing, if someone creates a zone, we'll capture the details.
See the screenshots below for the audit settings for zone deletion:
To audit for DNS record deletions, you should target the desired critical record objects. If you set this auditing up on all records, due to the churn of DHCP and DynamicDNS, you'll likely have a lot of Event Log volume and noise.
For Deletions via the DNS UI/tools:
Principal = Everyone
For deletions via ADSIEdit:
Principal = Everyone
Here is a screen shot of an audit event from a record deletion via ADSIEdit
The unique nature of AD-integrated DNS deletions.
When a 'typical' AD object is logically deleted, most of the attribute values are blanked out and it is 'moved' into the deletedObjects container. These changes are replicated across the DCs and then later, when garbage collection happens on each DC, the object is 'physically' deleted from the local AD DB.
DNS is a bit different, though.
When an AD-integrated DNS record is deleted via DNS tools, the AD object is not actually deleted from Active Directory (like a normal AD object). Instead, the dnsTombstoned attribute is set to TRUE on the object, and the record is removed from the in-memory cache of the DNS server and hidden from the MMC UI (you can still see it in ADSIEdit, though). Every day, at 2 AM, a DNS thread spins up and all records that are dnsTombstoned=TRUE and have been that way for 7 days are deleted from AD (like a normal AD object).
AD-integrated zones, on the other hand, behave like any other AD object when deleted. Most attributes are stripped off, and the zone object is moved to the deletedObjects container. Any DNS records within the zone are also deleted at this time, rather than going through the dnsTombstoned process.
For more in-depth info on DNS deletion and scavenging, see these great posts:
So now you can answer the W questions for DNS disasters … but wouldn't it be nice if we never were asked those questions to begin with?
Until then, Happy Columbus Day from Hilde, Brent Whitlow and Bryan Zink :)