Recently published on Microsoft TechNet (30 October) were the Software boundaries and limits for SharePoint 2013. Some noticeable adds are in this guidance so please take the time and familiarize yourself with the changes. Some of the guidance however did not change and a couple in particular I want to reference due to the fact on pretty much every engagement I go on with a customer these areas are not adhered to and in some cases blown completely out of the water.
List view lookup threshold
8 join operations per query
Threshold
Specifies the maximum number of joins allowed per query, such as those based on lookup, person/group, or workflow status columns. If the query uses more than eight joins, the operation is blocked. This does not apply to single item operations. When using the maximal view via the object model (by not specifying any view fields), SharePoint will return up to the first eight lookups.
List view threshold
5,000
Specifies the maximum number of list or library items that a database operation, such as a query, can process at the same time outside the daily time window set by the administrator during which queries are unrestricted.
To give an example of what I see customers do:
Notice where I highlighted.
These two setting changes will provide you with the opportunity one day to bring your farm to its knees and in most cases will also cause you to pick up the phone and contact Microsoft to open a case. This is a case that can easily be avoided.. How? By reading the Software Boundaries and Limits for SharePoint 2013. We put this guidance out for a reason and the teams aren’t just randomly selecting numbersout of the air and filling in the charts. We do extensive testing on our products with many different hardware configurations that we expect our customers to run under. These boundary numbers are areas where wewill see performance degradation occur. Also notice that some of these are soft boundaries and some are hard. The Hard boundaries are supportable stop points and should not be exceeded. The Soft boundaries fall in line with the example above. This is a throttling limit we set. You can adjust it if you like but you will face issues down the line. Depending on your hardware you may not run into it for some time pastthe limit. But I can guarantee you that in some time you will run into an issue here.
So let’s go through an example of what this customer faced. The background here is that I was there for a week performing a Microsoft RAP (Risk Assessment Program) and this is one of the test cases that werun.
Scenario
I run through the test cases and I am starting my analysis of the FARM’s Health and Risks… I come across a case that states that the “List View Threshold is set to a value greater than 5,000” At this point I look into the details and what I’m looking for are the number of list items shown. In this case one list had 60K items. Remember we are not concerned about the number of list items… were concerned with the View. I then have the customer open up Central Admin and I look at the Resource Throttling section for the Web Apps in question and low and behold I find this was changed. Customer stated that they made this changeearly on in the deployment but didn’t know if anything would be affected. These types of findings allow us to amplify the importance of RAPs where we provide proactive side by side discussions withcustomers. Many times these discussions open interesting findings that we can hopefully correct so as not to have the customer face issues in the future. During this conversation the customer mentioned they had an outage a few months ago and they were about to open a Premier Support Ticket but it suddenly normalized. This was all I needed. This little snippet of info allowed me to prove that this issue is real and was experienced (though the customer never found out root cause)
Changing these settings will cause an issue known as SQL Escalation Lock. Basically what happens is that you can have a very large list view in place. You’re a member of a team site and you openup that large list. You’re freely viewing the items in this list and notice no issue. Other users however are not so lucky. You’re opening up that list in an exclusive mode where others will receive an error when trying to view. But this is just one list. Who cares right? This gets better… follow along. You are done viewing this list and close it. This is where the real problem occurs. What happens now is that your SQL Processors are probably spiked and remain spiked. Performance is now tanked. Why? SQL is trying to remove the lock and depending on how large that list really is... it can take some time for it to heal itself. This is a SQL thing… not so much a SharePoint thing. SQL Queries are handled much better with the numbers we state in the Boundaries list. In time however the CPU’s will normalize but in many cases the customers panic and call an outage (which this is) but we can avoid this from happening… read on.
What to do about this?
Two methods are recommended
In MOSS the first option was the preferred choice. However in SharePoint 2010 and 2013 what we see is you still start off with better performance with option 1 but once you start to hit a list size of 100K it starts to level off to same performance results. When I work with my customers depending on how many lists are affected I generally will tell them to go with option 2 because its pretty simple to implement. Option 1 will take quite a bit of work moving items into their desired folders and there will need to be some planning with regards to naming conventions.
Don’t believe me? Have a look at the info in this guidance http://technet.microsoft.com/en-us/library/cc262813.aspx#Throttling (for those running SharePoint 2013 this guidance is still relevant)
Hope this avoids issues for someone in the future
In SharePoint 2010 we introduced FIM which acted as a broker of sorts when bringing profiles from AD into SharePoint. One of the key reasons for this add was to allow companies to not only pull AD info into SharePoint but also to push information from SharePoint back to AD. This was a key change from Microsoft Office SharePoint Server 2007 or MOSS 2007 as we generally called it. The synchronization method was a very simple one way pull from AD which didn’t allow a lot of flexibility.
One of the pains that were felt with User Profile Service Application (or UPSA, UPA, Dir Syncer or whatever folks like to refer to it) was that it required more steps to configure than just going into the UI andclicking a couple buttons as we Admin types are generally fond of doing. Early on in the process my buddy Spence wrote the manifesto on how to properly create this. There was a lot of bad guidance floatingaround out in the webosphere on how to do this and pretty much all of them were wrong. As much as it pains me to admit as I don’t want to inflate an ego but one of the questions I would always askmy customers when they deployed SharePoint 2010 went a little like this:
Me – Oh great it looks like you have every Service Application provisioned (this to me 99% of the time pointed me to the fact that they used the FCW (Farm Configuration Wizard) to deploy the farm) Moreto come from that another time but this is a bad plan for Production Farms. Fine for Dev or Test but Prod….Please don’t do this.
Customer – Yep it was simple but I can’t seem to get the User Profile thing working
Me – did you follow any guidance out on the web for this?
Customer – No I just let SharePoint handle it
Me – rubbing my hand through my hair wondering what a proper response would be while continuing to be a Trusted Advisor……
Me – You know we have a couple excellent documents out there that detail what needs to be done here to configure this. Namely TechNet or my buddy with the sillypurple page In fact if you followthis guide step by step you will always have a successful experience in deploying UPA.
Getting back on track to the point of this post is to inform you that those guides are still fully relevant today in SharePoint 2013. There is however one exception to this. Along with the previous mentioned method to Profile Synchronization we have also included the previous method of import…It’s called Active Directory Direct import and think of this as the lightweight import method.
The method of implementing this is not right in front of your face though. When I first started looking at this I would have assumed that they would add this functionality at the point where you create your Profile Synch but this lives in a different area. I will highlight both SharePoint 2010 and 2013 in the following screenshots to show this:
SharePoint 2010 (notice the arrow pointing to Configure Sync Settings)
SharePoint 2013 (Notice that the screens look remarkably similar)
SharePoint 2010 (Configure Sync Settings)
SharePoint 2013 (Configure Sync Settings)
As I am illustrating here this is where you would differentiate between the two methods. By default we are setting this for FIM but if you choose to go with AD Direct you would set this at this location.
One additional thing to note here is that in the previous version we had to have Replicate Directory Changes set via Delegate Control in Active Directory. This is still a necessary step for either method you choose here.
Cheers!
Additional Note: Spence informed me that he recently put out an updated article that can be found here
http://harbar.net/archive/2012/07/23/sp13adi.aspx
<A Microsoft PFE’s Note from the Field>
So in my day to day SharePoint support endeavors one of theclassic cases that come in from time to time is customers trying to implementKerberos into their Farms. One of theissues I see pop up a lot is duplicate SPN’s.
The classic method (via the command line) is to run thefollowing command to create an SPN:
Setspn –a http/<FQDN or Netbios Name> Domain\usercredential
For example Setspn –ahttp/connect.sharepoint.comsharepoint\content1
This would create an SPN to be used to create a validKerberos ticket for the http/connect.sharepoint.com site. Easy stuff here and you can verify this by going into ADSIEdit and openthe properties for the Content1 Service Name.
But how do we get the issue with Duplicate SPN’s. Usually this comes down to poor planningbefore you turn on your computer. Sayas an example (and one I see often) you didn’t think this through and plan yourAuthN properly. Or say you just made asimple “FatFinger” or had a “Brain Cramp”… it happens. For illustration purposes lets go with theidea that you created the above mentioned SPN but for some reason you tried torun the Setspn command again using a different account. Voila! that’s where we are in trouble. In my example I created two SPN entriespointing to the same site but I used two different service accounts. SPContent and SPContent1. As you can see in the screenshot below I getan Event ID 11 thrown which if you can read the text is basically telling meexactly what I need to know to go forward and fix the issue.
So is there a way to verify this before I go and run thecommand? Absolutly. If you enter the command Setspn –x you willget a list of any duplicates that exist in your environment.
Let’s take this one step further and combineboth the –A and the –X into a single FatFinger Proof method of creating our SPN’s
Operation ABORTED!!! So if we look at the above output were shown the same information as theSetspn –x but we are also combining this with the ability to add an SPN. In both of these outputs you can clearly seethat in my demonstration I tried to create an SPN for the same sitehttp/connect.sharepoint.com but I used separate accounts. Also note that I can use the same accountfor multiple hosts if I so choose.. in this case im using Spcontent1 for bothhttp/connect and http/my.
So in a perfect world when you are doing this work and you don’thave any duplicates you can use either the setspn –x initially to see if anyduplicates exist or do what I always do… use the setspn –s and cover everythingin one sweep. In the above example ifthere were no duplicates the process would successfully finish and presto changoyou have your SPNs created
Hope this helps someone out there J Cheers!
One question I get asked a lot from customers is aroundpatching their SharePoint environments. In general you were always able to install the bits on all your serversin any order you felt like going.. the real question or issue I have found isthe completion step which is running the Configuration Wizard. Some customers went so far as to run theinstaller to load the bits on their boxes but may have only run theConfiguration Wizard on one box… or in some cases maybe in a larger farm theysimply omitted a box.. this puts your environment at risk.
In SharePoint 2010 I still like to hit my main server that’srunning CA first with Config Wizard (or PSCONFIG if you choose thatmethod) But there is nothing stoppingyou from going to the other boxes and kicking off the Config Wiz while theother is running…. What will happen is that the other boxes will go into alocked state for the upgrade. Once onebox completes the others will in sequence of starting the Config Wiz runthrough the 10 step upgrade. As you can see in the screenshot below thisparticular box is waiting for the lock to release to continue.
The one real take away I need to make is to ensure in a multi-server environment you run this on each server.
Building a brand new SharePoint 2010 Environment? Want to decrease some of the time for makingsure your environment is up to date with regards to versioning? Slipstream your build.
In the following section I am going to detail how I build myslipstreams… there are other posts out there that will be similar and may even includeshortcuts such as posting the bits directly to the update folder but I like tobe a little more hands on and in reality it does not take a lot of effort.
New Environment
Download the SharePoint 2010 RTM Bits (http://technet.microsoft.com/en-us/evalcenter/ee388573)
I shorten the name of the downloaded file before I begin tosomething like SP10.exe and SFSSP1 thiswill make it a bit easier on the typing.
Download the SP1 Bits.
Although technically not necessary I will still download andincorporate both the SharePoint and the SFS SP1 Bits into my slipstream.
http://www.microsoft.com/download/en/details.aspx?id=26623
http://support.microsoft.com/kb/2460058
Same as with the RTM bits… I will shorten the names of thesefiles to help with typing. SFSP1.exe andOFFSP1.exe
Download the CU’s
http://support.microsoft.com/kb/2536599
http://support.microsoft.com/kb/2553931
Lets get busy
So this was the hardest part of the operation. What I do next is create four folders and extract the bits for each into them.
SP1 – Folder names OFFSP1 SFSSP1
Command I run: sp1.exe /extract:c:\OFFSP1 (where sp1.exe is the renamed executable and OFFSP1 is the folder I created and am extracting to.
Sfssp1.exe /Extract:c:\SFSSP1
I will do this for the Cumulative Updates as well.
Once this is complete the SharePoint 2010 Installationfolder has a folder called updates. Iam simply going to copy and paste the contents of each of the folders I extracteddirectly into Updates.
This concludes the Slipstream procedure. Once complete you can go forth and install. Your environment will be up to date and you wont have to go through the process of individually installingpatches and then remembering if you hit every server in the process… It happens (unfortunately very often)
Additional reference information can be found here at my buddy Spence’s site http://www.harbar.net/archive/2011/06/30/327.aspx
I wanted to post out a couple things that I have found recently that may cause some confusion when trying to integrate an Existing SharePoint 2010 Environment with FAST Search.
If you already have an existing Search SSA you can safely remove this once you install and configure your FAST environment. This SSA is NOT needed. FAST has two seperate SSA's:
FASTContent - This SSA will hold the Content Source for all the sites that will be crawled under the FAST Search Engine.
FAST Query - This SSA in simply responsible for the sps3:// (profile imports)
Now the confusion will arise when you create a new web application and expect it to go directly into the FAST Content SSA Content Source. It does not. It will go directly to the FAST Query SSA Content Source.
What does all this mean? RIght now you need to ensure that any sites that you want to crawl live inside the FAST Content SSA. You will need to manually move any sites you create from the FAST Query to the FAST Content Content Source.
Just something to keep in mind. Not a huge deal but will be another step for Admins to put on thier checklists.
Way too much debate out in Twitterville and through other folks that are just flat out wrong. Why wrong? Well in a test lab environment I have no problem with this but folks tend to get lazy and that is where you run risks in your production environment.
Now im not going lament on the whys as this was finely detailed by Spence Harbar and let me quote from his post
"What is the issue? Windows Server 2003 SP1 introduced a loopback security check. This feature is obviously also present in Windows Server 2008. The feature prevents access to a web application using a fully qualified domain name (FQDN) if an attempt to access it takes place from a machine that hosts that application. The end result is a 401.1 Access Denied from the web server and a logon failure in the event log. Unfortunately 401.1 is not really helpful as this error code means there is a problem with the user credentials. Of course, the HTTP spec doesn't know about security features in a vendor's implementation so there can't be a HTTP error code for such a feature. This can lead to much banging of the head on the desk. It's one of numerous causes of the 401.1 which are nothing to do with invalid credentials (e.g. attempting to use Kernel Mode Authentication with domain account in IIS7). What this means is that when you browse a SharePoint Web Application which uses a fully qualified domain name from a WFE in the farm you will get a 401.1. This is very annoying on a development box, or when testing locally, or in other SharePoint specific scenario "
"What is the issue? Windows Server 2003 SP1 introduced a loopback security check. This feature is obviously also present in Windows Server 2008. The feature prevents access to a web application using a fully qualified domain name (FQDN) if an attempt to access it takes place from a machine that hosts that application. The end result is a 401.1 Access Denied from the web server and a logon failure in the event log.
Unfortunately 401.1 is not really helpful as this error code means there is a problem with the user credentials. Of course, the HTTP spec doesn't know about security features in a vendor's implementation so there can't be a HTTP error code for such a feature. This can lead to much banging of the head on the desk. It's one of numerous causes of the 401.1 which are nothing to do with invalid credentials (e.g. attempting to use Kernel Mode Authentication with domain account in IIS7).
What this means is that when you browse a SharePoint Web Application which uses a fully qualified domain name from a WFE in the farm you will get a 401.1. This is very annoying on a development box, or when testing locally, or in other SharePoint specific scenario "
Ok so we have some background info the one thing my buddy doesn't do is show us how. Now the crux is that in http://support.microsoft.com/kb/896861 Microsoft details two such fixes for this. I am going to screenshot the right way to do this. First off we need to add the following registry key to kick this off
Ok so we have some background info the one thing my buddy doesn't do is show us how. Now the crux is that in http://support.microsoft.com/kb/896861 Microsoft details two such fixes for this. I am going to screenshot the right way to do this.
First off we need to add the following registry key to kick this off
In the screenshot above under HKEY_LM\system\CCS\Services\Lanmanserver\param we will create a dword DisableStrictNameChecking. Add a value of 1 to this new entry,
Exit Registry and reboot your box
Reopen Registry and nav to HKEY_LM\System\CCS\Control\LSA\MSV1.0 and create the following key as shown below
Once there simply open this Multi-String Value and enter the sites you want included… ie your SharePoint sites J No need for URLs here.. simply type in (for this example) connect.contoso.com on a separate line your next site and on and on down the line. The beauty of this method is that once you add this key you wont have to reboot your box after adding these entries. So hope this post along with Spences stops the silly questions and even more so… wrong answers folks are following
Once there simply open this Multi-String Value and enter the sites you want included… ie your SharePoint sites J
No need for URLs here.. simply type in (for this example) connect.contoso.com on a separate line your next site and on and on down the line.
The beauty of this method is that once you add this key you wont have to reboot your box after adding these entries.
So hope this post along with Spences stops the silly questions and even more so… wrong answers folks are following
So as most of you know im a pretty big advocate on Virtualizing SharePoint and do a lot of talks on lab setups and such. Way back when Server Core came out I gave it a spin but found a lot of the toolsets to be lacking and frustrating. Windows Server 2008 R2 has made me jump back into this and in this post I will detail how I am now setting up my SharePoint 2010 Demo Labs.
For starters this article is not really going to talk about SharePoint 2010 but what I do want to talk about is the underlying architecture that makes SharePoint tick. One of these necessities is Active Directory. I am fully aware that you can install your test farms using local accounts. If that is how your building your labs out you can refer to Neil Hodgkinson's post on how to accomplish this. It is definitely an option to use but for my purposes I like to have AD available to be able to pull the areas that you can only get with an AD / SharePoint instance… one classic case would be UPS (User Profile Service) which in turn has tie ins to Social Computing aspects. Ok enough rubbish lets get on with it.
So the real beauty of running a Core Server with Active Directory is that you can do the following:
1. Run your AD instance with very minimal memory ( 256MB… how's that for minimal)
2. If you spin up multiple labs this DC is completely reusable. Once you set it up you can spin up as many labs as you want pointing back to that Directory Server.
Ok so some of you are thinking well hell Fox…. Its just as easy to spin up a normal box and run DCPROMO… why do I need this? Well you don't…. This is simply a personal preference of mine and one benefit I see is that you don't have that OS overhead you get with a full blown box. Oh… and might I add that it takes less than 5 min to build a box from start to finish? When I say build I am talking installing the CORE OS. The next steps take a little longer but hopefully some of the information I provide here will quicken the process.
Before we go further I mentioned earlier that the box will run pretty light with memory. What I recommend is to give it a bit of memory ( 800 – 1GB) while your doing the build out process. Once all is done you can throttle back the machine.
Steps to Install Active Directory
Im not going to detail how to build out Windows 2008 R2 Core as its pretty simple… point to your ISO file and start the box. When you have the option to choose your OS you will go with Core and off you go.
Once the box is complete and reboots you will be greeted by a cmd.exe shell. So some of you who are reliant on the GUI (I can be accused of this sometimes) you may find yourself scratching your head…. Hmm dir… hmmm cd…hmmm so on… yep those commands are all still there, but what was missing in Server 2008 Core is a command called "sconfig" Go ahead type that in.
So lets walk through this shell window a bit.
Option 1 Domain/Workgroup will obviously be something to avoid here for a bit as once you run DCPROMO it will automagically assign this role so leave this alone.
Option 2 You definitely want to make a change here. I tend to always go with DC1 for my builds but name it what you want. This will cause a reboot to make this change
Option 3 Add Local Administrator - So when you spin up the box the first time it will already log into the Administrator account. If you wish to add another account this is the section you will make that change.
Option 4 Configure Remote Management - This section needs some screenshot love and explanation
So entering 4 opens the configuration portion. You probably WILL want to enable options 1, 2 and 3… If you're a PowerShell junkie option 2 is a must J
Adding these options will allow for remote management from GUI bases boxes (which we will get into in a bit)
Im going to skip through some of these other options as there pretty self-explanatory but let's look at Network Settings Option 8
To get started you will need to select the Adaptor. Since I only have one assigned to this box I will choose 0
So on first entry here you will have your assigned addresses but we cant have that can we. Options 1 and 2 will allow you to assign proper static addresses.
Ok so there are the basics for configuration… Miss the GUI yet? Well if so there are other ways to manage this server directly from the console.
When hashing this set up over with Spence he turned me over to this site to grab another tool http://coreconfig.codeplex.com/ The owners of this tool did a pretty good job documenting and screen shooting this so I'm not going to spend time here. Have a look for yourself.
Creating your AD Instance
So most of you who have built out an AD server are familiar with running DCPromo.exe. Don't be fooled by typing this in and thinking its going to work… We need a bit more. You will need to provide an answer file to make this work. Below is the answer file I created and you can grab the txt and place it in a file called.. wait for it… wait…. Answerfile.txt
[DCINSTALL]
InstallDNS=yes
NewDomain=forest
NewDomainDNSName=Contoso.com
DomainNetBiosName=Contoso
SiteName=Default-First-Site-Name
ReplicaOrNewDomain=domain
ForestLevel=3
DomainLevel=3
DatabasePath=c:\NTDS
LogPath=c:\NTDS
RebootOnCompletion=yes
SYSVOLPath=c:\SYSVOL
SafeModeAdminPassword=pass@word1
So if you were to create a new domain named Contoso and enjoy using the standard password that Microsoft loves to use all you have to do is copy this file exactly and use it. To implement this we will run the following command
Dcpromo.exe /unattend:c:\answerfile.txt (this is assuming you placed the answerfile at the root of C:… yeah I know im insulting someones intelligence but I do get some odd questions sometimes J )
Once you enter this the box will go through the process of creating an Active Directory instance for your lab.
DNS
This is a big one you don't want to overlook. So this is again something that can be run from the command line using dnscmd.exe.
You will want to define your SharePoint Web Applications that you want to set host headers for. In my example I am setting DNS A Records for the following Web Apps.. Connect and My
dnscmd dc1 /recordAdd contoso.com connect A 192.168.1.61
dnscmd dc1 /recordAdd contoso.com my A 192.168.1.61
So for the IP address this will be pointing to SharePoint.
Note: In my talks I generally mention the fact that you create labs for different scenarios and because memory is going to be an issue to consider you will normally only have one lab scenario running at a time. As long as the other labs are shut down and your using one at a time you can most certainly reuse IP addresses. This way you will not have to go back and modify DNS.
IMPORTANT NOTE/REMINDER: Im talking test labs here folks. Do not do this in Production if you wish to keep your job.. Boss to you… YOU DID WHAT????
Final Stuff
Going forward you will want to add users and service accounts to your domain. You can do this via command line on the server or find a script out on the web to accomplish this if you have a lot of users you want to toss in but you can also log into another server to do this. You will need to add the Domain Services Role to the server but the reason for this is so you can remotely admin the DC. It will fail if you just open it and click Active Directory Users and Computers so make sure you right click that section and select "Change Domain"
I think ya'll can take it from here.
Once done shut down the AD box and throttle back the memory.
In Conclusion
So where do we go from here. SQL Server on Core? Heck yeah… check out Dan Browns post on this.
Again this is just one part of creating a lab environment and it's the one im using consistently. SharePoint 2010 is a huge gobbler of Memory so I want to ensure I have enough on my local box to give to it. Throttling this back and running lean and mean definitely lets me keep my SharePoint box running at more than a snails pace J
Cheers