Ten Tips and Tricks for Server Baselines

Ten Tips and Tricks for Server Baselines

  • Comments 1
  • Likes

Happy Friday AskPerf!  One of our Technical Account Managers pinged me a few weeks ago and asked if I could put together something on how to do a server baseline.  After thinking about how best to do this for a while, I realized that trying to create a one-stop shop for server performance baselines goes well beyond a blog post and into the realms of entire books!  But, that doesn’t mean that we don’t have some good tips for you …

So, to start with – what exactly is a baseline?  Simply put, a baseline is a picture of how your server normally operates.  Having a baseline is incredibly useful you’re doing capacity planning, performance-related troubleshooting, or even disaster recovery planning.  Now, a good server baseline encompasses far more than just running a Performance Monitor for a couple of hours and then calling it a day.  All the tips below can be implemented using Microsoft tools that either come with the server or are free to download.  So, let’s get started …

10. One Baseline simply won’t do:  OK, you’re already thinking … what?  What are you talking about?  Server Performance is affected by time.  Think about it – Monday mornings (well, probably every morning) at 9:00 am are probably when your domain controllers are busiest trying to resolve authentication requests from users logging in after the weekend.  File servers have probably been relatively dormant over the weekend as well.  Terminal Servers – well, that goes without saying … doesn’t it?  By the same token, getting a baseline at the end of the day isn’t a bad idea either.  We just talked about logon / logoff issues on Terminal Servers due to profile issues.  Do the same “slowness” problems occur on Monday morning and Friday afternoon?  You’ll definitely want to get an understanding of server performance during those peak times, as well as at different times during the day – and even during your business cycle (such as year-end processing).  Which brings me to …

9. You’re a 24x7 shop.  Backups and Data Analysis Jobs count too:  We’ll often hear from customers reporting that backups are “slow”.  Well, “slow” is a relative term.  If you only capture your baseline data during business hours, then you may be missing out on rich data gathering opportunities during off-hours when – let’s face it, some of your most important business functions take place.  That’s money time when it comes time to do your Disaster Recovery exercise.  No good backups means no Disaster Recovery capabilities.  In addition, if you’re responsible for servers that do after-hours data processing (SQL Servers primarily is what I’m thinking of here), then you’ll definitely want to get an idea of how long those jobs take to run.  Make sure you pay just as much attention to what happens when you’re not in the building as you do when you are.  Remember, there are lots of variables when it comes to performance, and thus the importance of …

8. Gathering trending data:  I mentioned getting baseline information at different times of the day – as part of your baseline process, consider taking longer performance monitor captures with longer intervals so that you can see the complete picture over a number of days etc.  This is the type of data that really carries weight when it’s time to go ask for more money as part of your capacity planning!  And that provides a great segue into the fact that …

7. Your baseline is more than just your server: OK, given that this is a post about server baselines, you’re probably wondering where I’m going with this.  Just as you need to account for different time slices, and trending data, you also have to account for your entire infrastructure.  Be holistic in your baseline captures.  If you have SAN’s, have the SAN vendor review the performance of their hardware as well as LUN configuration on a regular basis – not just when something isn’t acting quite right.  In addition, keep an eye on your network health.  Sometimes issues that seem to be server performance problems aren’t really server-side problems.  Which is why we have to remember that …

6. Applications need love too: We’ve discussed hardware, the network and the operating system.  But, applications sometimes have their own performance quirks.  If the application is one that was developed in-house, then you can get your hands on the developers (usually) and get some idea of how their application should perform.  For 3rd party products this may be a little more tricky, but there is no harm in calling up your ISV and trying to get some idea of acceptable application performance – especially when you’re talking about n-tier type applications with lots of complexity.  Knowing how an application should be performing versus what it’s doing in your environment can help you identify potential issues before they become all-night / all-weekend conference calls during year-end processing.  Part of being successful with the last three or four things I’ve mentioned is that you’ll need to …

5. Be Inquisitive!  Asking questions isn’t a bad thing:  There’s a saying that “the only dumb question is the one you don’t ask”.  Whether you’re an IT Admin responsible for the operating system, an application developer, a DBA or the Network guru, more knowledge is not a bad thing.  Before I started at Microsoft I worked in a very compartmentalized IT environment.  It was “the way things had always been done” – namely that the network guys didn’t readily share information with the OS team, who didn’t always collaborate well with the application implementers … well, you can see where this is going.  Not only did this mean that no-one really had a clear picture of what was going on even in a general sense, but if (when!) something went wrong, it took twice as long to make progress because either no-one knew who to call, or how to explain the problem in terms that made sense to other groups.  Sharing knowledge doesn’t mean that you’re sacrificing job security – you’re not turning over the keys to the kingdom!  It means that you’re collaborating and becoming a cohesive team!  Beyond the obvious technical knowledge that passes back and forth, the human relationships that you forge will make your life easier!  And in terms of holistic performance monitoring and baselines, it always helps to know just who to go.  OK, let me hop off my soapbox now (yes, this is one of my biggest peeves about IT environments!) and move on to why …

4. Documentation matters:  I’m sure everyone has a plethora of horror stories about poor / non-existent documentation and the resultant nightmares.  In complex enterprise environments, documentation reigns supreme.  When you’re talking about historical performance and capacity planning in particular, it’s all about the numbers.  If you can’t prove that you need something, the odds are you’re not going to get it.  Keep your historical baselines, and make sure you can put your hands on your change management history.  That’s almost Troubleshooting 101 - “What changed?” – but you’d be amazed at how many folks can’t find out how / why / when something changed in their environment.  And that’s when we’re troubleshooting – now add Audit and Disaster Recovery to your list of considerations and if someone isn’t already keeping tabs on … well, everything – then it’s time to start.  Now, if you have all of this all set up and ready to go, let’s turn our attention to why …

3. Problems aren’t always problems:  Deep, huh?  But, look at it from this perspective - server performance doesn’t just get worse without a reason.  The underlying causes may certainly be problems – bad hardware, problematic application update, non-optimized code etc.  But – problems are also opportunities.  Slow performance may also be caused by excessive load, slower hardware (different from bad hardware!), and again – non-optimized application code.  If you can provide evidence as to how performance is relatively poorer when compared to weeks, months or even years ago then you’ve provided a business reason for that server upgrade or application review - especially if you …

2. Don’t take the humans out of the loop:  I remember when the movie WarGames came out many, many years ago (and yes, I know I’ve just dated myself!).  One of the most telling lines in that movie was when someone said, “Take the men out of the loop”.  Movies like The Matrix and The Terminator were built off that particular sentiment!  There are plenty of tools out there that can monitor your servers and tell you when something’s wrong.  What they can’t necessarily do is tell you WHY.  Your monitoring software can’t intuitively make the connection between your frontend web server being slow thanks to caused by your middle-tier server not processing requests quickly which was caused by network latency due to slow authentication that was caused by the domain controller being overworked and trying to satisfy all of the LDAP requests caused by the guys doing a complete Active Directory dump as part of their server migration project.  It takes “the little gray cells” (as Hercule Poirot was so fond of saying) to put the entire puzzle together.  And last, but by no means least, that means you have to …

1. Keep it FRESH!:  I can’t stress this one enough.  That server baseline you took two years ago may not be of too much value today.  A number of different things have undoubtedly changed – application updates, network updates security patches, increased load, updated drivers.  All of these things contribute to subtle (and sometimes not so subtle) variances in your server’s performance.  For your mission-critical servers, you should be staying on top of your server’s performance – every minute that the server isn’t responding is a minute you might not be processing a sale, or closing a deal.

OK – that’s it for today’s post.  Quirky?  Perhaps.  But hopefully there are some nuggets here that can help you in your IT Adventures.  I’ll go over some standard Performance Monitor counters and how you can use them as while doing your baselines in a future post.  Until next time …

- CC Hameed

Share this post :


Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Definitely good guidelines to follow for building Baselines. Thanks