The blog of Wictor Wilén
The blog of Wictor Wilén - SharePoint Certified Architect, Certified Master and Most Valuable Professional
RSS
-
SharePoint 2013: Intelligent Promoted Results and Best Bets
Introduction
The search engine and search experience in SharePoint 2013 has been totally re-written, since its predecessors. In SharePoint 2010 we had something called Best Bets or Visual Best Bets if you worked with FAST for SharePoint. A best bet was a promoted result that was triggered by one or more keywords, used by the search admins to promote certain links or banners for specific search queries. In SharePoint 2013 this is now called a Promoted Results and the procedure of creating them is different and so much better – there’s more ways to trigger on, more ways to render the results etc, but the actual shown result isn’t that smart, until now…
In this post I will show you how to create an even smarter and more intelligent Promoted Result – a best bet that actually uses the search query to do something interesting with. In this sample I will let users enter simple math questions and then we let the promoted result calculate the math question for you (just as the big search engines on the interwebz does).
Creating Promoted Results aka Best Bets
A Promoted Result is created either on Search Service Application, Site Collection or Site level using the Search Query Rules option. To create a new Site Collection scoped Promoted Result you navigate to Site Settings > Search Query Rules. Next thing to do is to choose which scope to create this Query Rule for, choose Local SharePoint Results in this case. You have the option to further narrow down the Query Rule using segments and topic categories, but that’s not relevant for this exercise. Next thing to do is to click on New Query Rule. The Add Query Rule page contains quite a few options to make sweet things happening. First we have to give it a name and then we have to specify the query conditions. Query Conditions is what is used to trigger this promoted result. In order to trigger on our “mathematical” question we need to use a regular expression. To use a regular expression as the Query Condition you have to select Advanced Query Text Match and then write a regular expression.
I’m no regular expression savant, but this regular expression works for this blog post. Anyone with a cooler or smarter or more complex regex – feel free to post it in the comments.
\d+\s*[/\+\-\/*]\s*\d+
Using a custom page as the Promoted Results
Next up is to add the Promoted Results Action to this Query Rule. This is done by clicking on the Add Promoted Result further down on the Add Query Rule page. When adding a promoted result you will be asked for a Title, a URL and a description. In our case the URL is of interest – we want to render a page containing our logic to calculate the math query.
I let the URL point to a page (an .aspx page we’ll create in a few seconds) and then I pass two query string parameters to it; IsDlg=1 to get rid of the chrome in the page and query={SearchTerms} to pass the actual query to the page. I also check the check box Render the URL as a banner instead of as a hyperlink so that the promoted result will be rendered as an iframe instead of a link.
Seem simple, doesn’t it, but wait – this will not work until you followed all the steps in this post. There is no thing out of the box that allows you to pass the search query to the Visual Best Bet.
Now click Save on the Add Promoted Result page and then Save on the Add Query Rule page.
Creating the custom promoted results page
Now, before we test this (and it also takes a couple of seconds before this promoted result can be used – don’t ask me why!) let’s create our page that should calculate the result from the query. For this demo we create a fairly simple .aspx page with a JavaScript section and a small HTML snippet, as shown in the snippet below:
<%@ Page Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage, Microsoft.SharePoint,
Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
MasterPageFile="~masterurl/default.master" Language="C#" %> <asp:Content ContentPlaceHolderID="PlaceHolderAdditionalPageHead" runat="server"> <script type="text/javascript"> function getQueryStringValue(name) { var qs = window.location.search.substring(1); var pairs = qs.split("&"); for (var i = 0; i < pairs.length; i++) { var pair = pairs[i].split("="); if (pair[0] == name) { return pair[1]; } } return ''; } _spBodyOnLoadFunctions.push(function () { var query = getQueryStringValue("query") try { document.getElementById("result").innerHTML = eval(query) } catch (e) { document.getElementById("result").innerHTML = "error" } }); </script> <style> #s4-ribbonrow { display:none; } </style> </asp:Content> <asp:Content ContentPlaceHolderID="PlaceHolderPageTitleInTitleArea" runat="server"> Intelligent Best Bet </asp:Content> <asp:Content ContentPlaceHolderID="PlaceHolderMain" runat="server"> <h1>SharePoint Search Calculator</h1> <h2>The result is: <span id="result"></span></h2> </asp:Content>The first section contains a simple JavaScript that extracts the Search Query from the query string and then uses the JavaScript eval() method on it to “calculate” the value. In the PlaceHolderMain I added a span element which is where the calculator writes the result.
So let’s upload this page to the Documents library in the Search Center – which was the location I specified when creating the Promoted Results. If we now search for something that should be triggered by the Query Rule you will see the page rendered as a banner, but it will not calculate the query as we expected.
Passing the search query to the custom promoted results page
So, why aren’t our result rendering properly? It all comes down to that the default Display Template for Best Bets/Promoted Results does not pass the search query to the iframe. In order to fix this we need to modify this Display Template. The Display Template that we need to edit is the Item_BestBet.html file.
First we add two lines, directly following the var isVBB = … line, like this
var isVBB = ctx.CurrentItem.IsVisualBestBet; var url = ctx.CurrentItem.Url url = url.replace('{SearchTerms}',ctx.CurrentItem.ParentTableReference.Properties.QueryModification)The code is crafting the URL to use in the iframe by retrieving the URL from the CurrentItem and then replacing the text {SearchTerms} with the actual search terms used.
Next we need to modify the line where the iframe is rendered.
<iframe id="_#= $htmlEncode(id + Srch.U.Ids.visualBestBet) =#_" class="ms-srch-item-visualBestBet" title="_#= $htmlEncode(ctx.CurrentItem.Title) =#_" scrolling="no" frameborder="0px" src="_#= $urlHtmlEncode(url) =#_"></iframe>Modify the src attribute of the iframe element so that it uses our new url property instead of the default URL value from the current context. Save the file and go back to the Search Center…See it in action
Now we can take it for a test drive. Search for simple mathematical questions and see the results being rendered as a promoted result.
Summary
I’ve just shown you how to make the Promoted Results aka Visual Best Bets more intelligent by creating a Query Rule that promotes a web page that does operations on the search query. To get this all to work the secret ingredient was to modify the Display Template for promoted results to allow it to pass the search query to the promoted results page.
I can really see this being extended to provide the users with even more interesting “intelligent” promoted results.
-
SharePoint 2013 Central Administration Productivity Tip
Here’s a short post with a tip that I find very useful.
In many scenarios you have several SharePoint 2013 installations to handle – it might be different farms, production environments, testing, staging, development etc. Do you know which Central Administration you’re working in at the moment? They all look the same, SharePoint Blue, the regular Status Bar warning that you’re running out of disk space etc. Unless countermeasures are taken you don’t know what environment you’re in unless you take a look at the URL – which in many cases is just another server name and port. It’s very easy to make a mistake and make a change in the production environment instead of in the test or dev environments.
So, how do we keep track of what Central Administration site we’re actually working on at the moment? One way could be to change the theme of the Central Admin site. But, was the production CA red or was it the one with the dog in the background? I’ve got a better tip for you!
Here’s how I’ve done to keep track of the Central Administration sites. I take advantage of the Suite Link Bar in the upper left corner. By default it says just “SharePoint” – yes we know it’s SharePoint.

By modifying the Web Application property that controls this text we can easily change it to something more friendly and appropriate for the specific farm, like below.

It’s a very simple PowerShell operation to accomplish this. You just retrieve the Central Administration Web Application object, then update the SuiteBarBrandingElementHtml property and set it’s value to something that tells you which Central Administration site this is:
asnp microsoft.sharepoint.powershell $ca = Get-SPWebApplication -IncludeCentralAdministration | ` ?{$_.IsAdministrationWebApplication -eq $true} $ca.SuiteBarBrandingElementHtml = "<div class='ms-core-brandingText'>Central Admin: FarmA Production</div>" $ca.Update()You should leave the div element with the ms-core-brandingText class, to get decent formatting of the text, but inside it you can add whatever HTML you like (I’m thinking the marquee or blink tag…)..
That’s it. I hope I saved a few kittens from being slaughtered…
-
SharePoint 2013: Personal Site instantiation queues and bad throughput
In SharePoint 2013 the way Personal Sites (aka My Sites) are created have been totally rebuilt to support the new way of utilizing the Personal Sites. In this post I will go through how Personal Sites are provisioned, asynchronously, and bust a couple of myths about how interactive Personal Site instantiations should be “prioritized” and increase throughput.
Background
Personal Sites or My Sites were previously created “on-demand”. When a user went to his or hers non-existing My Site the provisioning started while the user waited for the site to be created, painfully watching the spinning animated gif. This worked fine in SharePoint 2010, and earlier, but now with SharePoint 2013 so much more are depending on the user having a Personal Site – everything from the social stuff in SharePoint 2013 (yes, not all SharePoint customers have yet wandered down the Yammer road) to the really great Office 2013 interaction (SkyDrive whatever). Requiring users to “manually” creating their Personal Sites is no longer a good option…
Note: this post is valid for SharePoint 2013 RTM – and hopefully some of the myths/bugs/features/stuff that is explained in this post will be fixed in upcoming releases…
Asynchronous Personal Site instantiation
In SharePoint 2013 the Personal Site is no longer created synchronously while the users wait for the site to be created. Instead all Personal Site provisioning are handled asynchronously by the Timer Job service (OWSTIMER.exe). So how does all this work? Whenever a user visits the My Site Host for the first time they are forced to create a Personal Site (depending on the My Site creation settings in the User Profile Service Application). The Personal Site instantiation are then added to one of possibly three queues (more about these in a bit). A set of timer jobs picks up the instantiation requests from these queues, creates the Personal Site and finally sends an e-mail to the user. The benefits of this new approach are many; the user can continue to do other stuff in SharePoint (real productive work instead of facetweeting for instance), the provisioning are not hogging the w3wp.exe process, we avoid performance bottlenecks, we should(!) see improved Personal Site creation throughput etc. There are of course drawbacks of this approach, such as it could potentially take some time before the personal site is created, since we have to wait for the timer job etc. But all in all this is something that is necessary (especially for the cloud) and something is very good.
The Instantiation queues
I mentioned that there are three queues for the Personal Site instantiation.
- The Interactive Request Queue
- The Second Interactive Request Queue
- The Non-interactive Request Queue
First of all – the difference between the Interactive and Non-interactive requests are basically that all “requests” to create a Personal Site through the web user interface are called interactive, while those requests coming from the Office client are non-interactive. The reason of having a second interactive request queue is that requests coming from the user interface – users actually wanting to facetweet – should have a higher priority and a higher personal site provisioning throughput.
Each queue are (by default) polled every minute, by three timer jobs – one per queue, Each Timer Job will retrieve five (5) items from the queue and process them one at a time. According to some (ahem, SPC12) presentations this timer job are executed on every WFE, but that’s not what happens in real life. The Timer Jobs are based on the SPWorkItemJobDefinition Job Definition Type. This is a really nice timer job implementation that has a queue per content database. In this case the queue exists in the Content Database where the My Site Host is residing. This also means since the lock type of the job is Content Database per Web Application – only one server will run this timer job. So, to sum it up – for each queue and every minute a total of 5 Personal Sites can be created, which means about 300 Personal Sites per hour per queue – if you have the hardware that can handle that! One might think now that since there are two interactive queues, we can get a throughput of 600 Personal Sites an hour, well…no…
The second (and useless) interactive queue
The idea with the second interactive queue is that interactive personal site requests should have a higher throughput, and that is a good idea. But unfortunately something went wrong when this was implemented. Stop reading now if you don’t like digging deep into SharePoint internals and just accept the fact that this implementation is flawed…
The three queues, or rather the three Timer Jobs are created per Web Application through a hidden feature called MySitesInstantiationQueues. This feature creates the timer jobs and also configures an object in the config database (SPPersistedObject) which contains a value of how many queues that should be used. The
funnything is that this is an internal definition (the SPPersistedObject) and it sets the number of instantiation queues to 1, not 2. This means that we’re hardcoded to only have one instantiation queue, and the second interactive timer job just don’t do anything since it’s queue is always empty – even though we could use reflection to modify it, but then we’re in unsupported territory. So there goes the idea of bumping up the number of Personal Sites created asynchronously.You can very easily see this in the ULS logs:
Also, if we take a look directly at the configuration object we’ll see that it is configured only for one queue:
So the idea of having servers provisioning up to 600 Personal Sites an hour we instead have 300 Personal Sites per farm per hour. Imagine a new corporate portal launch spiced with social features for 20.000 users and about 5% of the users visiting the site at launch. This will give us 1.000 users in the queue and a potential queue time of 3:20 hours, at least – not that great in my opinion. Just imagine if you did some marketing of your new launch and only a couple of more users tried to use the social features…
Well, the good thing is that we don’t have the web servers hogged with creating thousands of Personal Sites, prohibiting real work.
Increasing the throughput anyways…
This section should really be classified under the unsupported things you shouldn’t do, ever, unless you need to… But for a large Enterprise Social deployment using SharePoint 2013 it might be a necessity, to not piss the end users off by having them waiting four hours to get started using SharePoint 2013.
If you don’t want to provision all the Personal Sites in advance, which in most cases are a dumb idea, you could instead of just relying on the queues to provision the Personal Sites you could read the queue yourself and create the personal sites “manually”. You need to find the database where the My Site is hosted and then query the ScheduledWorkItems table for items. Query for where the Type is E94A6CAA-B0F5-4897-B489-585CA50C7803 (which is the id of the first interactive instantiation queue, the second queue has it’s own Type Id – but you will never see that in here :-)). Find those with the InternalState equal to 8 – those are the ones waiting to be processed. Using this information you can use PowerShell or similar to spread out the load on other servers to create the personal sites.
To create the actual personal sites you could use the CreatePersonalSite() method of the UserProfile object, this method will bypass the queue and immediately create the site.
Summary
By now you should be familiar with the process of how SharePoint provisions Personal Sites in SharePoint 2013. It’s a better approach than in previous versions. The idea of having multiple queues and allowing several servers to help out with provisioning the personal sites is good, but currently flawed. Hopefully this will get fixed in upcoming cumulative updates. As it is now it’s not working for a large enterprise rolling out SharePoint Enterprise Social, so you have to be very careful when planning a roll out of Enterprise Social in SharePoint 2013…or use Yammer…
-
SharePoint 2013 and Unified Access Gateway (UAG) 2010 Service Pack 3
Last week the Forefront team finally released Service Pack 3 for Forefront Unified Access Gateway (UAG) 2010. This is a long awaited release for us working with SharePoint 2013 and for those using non-legacy operating systems and browsers. In this post I will show you how to publish SharePoint 2013 Host Named Site Collections through UAG 2010 SP3 and consume the published site using Internet Explorer 10 on Windows 8.
What’s new in UAG 2010 Service Pack 3
Before diving into the interesting pieces let’s take a look at some of the new features of Service Pack 3. First of all, to be able to install SP3 you need to have your UAG at SP2 level – SP3 is NOT cumulative.
The most interesting features in SP3 are support for Windows 8, Windows Phone 8, Windows 8 RT and Internet Explorer 10 (classic and Metro) as client, support for the Office 2013 client and support for SharePoint 2013 as the published application. For SharePoint 2013 the URL Rule Sets has been modified to handle the new types of URL’s and request patterns used in SharePoint 2013, and new endpoint policies are added for SharePoint 2013. If you’re doing an 2010-2013 upgrade, you need to look over any custom URL Rules and policies. There’s also a number of improvements with regards to ADFS publishing.
Get all the nitty gritty details from the KB article: KB2744025.
Infrastructure
Ok, now you know what’s new! Let’s start publish our SharePoint 2013 sites through UAG. First of all before even installing UAG you need to think about your infrastructure, network, DNS and certificates (there it is again – certificates!). The picture below shows my demo environment. As you can see I have a UAG 2010 SP3 server with dual NIC’s (actually it have three – but we can safely ignore that). One NIC is connected to the Corporate Network and the other one to the External Network.
DNS
Before starting our publishing and SharePoint 2013 configuration we need to plan our DNS namespace. We need at least one publically available (that is a real) domain name. For this demo I will use three:
- trunk.corp.local – this will be used for my UAG Trunk Portal
- sp.corp.local – this will be a SharePoint 2013 site
- teams.sp.corp.local – this will be another SharePoint 2013 site
In order to make it efficient for the end-users I do prefer to have alignment of URL’s internally and externally. In this case it means that I have the second two domains above registered in the internal DNS as well. There is no need to register the Portal/Trunk domain internally.
Certificates
When publishing sensitive stuff such as SharePoint content you most likely would like to use HTTP over SSL, HTTPS, to encrypt the data flying throughout the internet. This is not only a good option to do externally, you should do it internally as well – even more now with SharePoint 2013 and the authorization flow over HTTP. Utilizing HTTPS internally also gives you the benefit of URL alignment for internal and external users and you don’t have to worry about sending an unreachable link. And we don’t have to do SSL termination.
What you essentially need is one or two certificates – one from a trusted root authority to use on the UAG server and one for SharePoint, which can be a local CA issued certificate. You could use the same though. You should use wildcard certs or SAN certificates here – so in the end it comes down to manageability and costs of certificates.In this sample I use one certificate, issued by my local CA. So any external clients, not joined to the domain will get a certification error (unless they have the CA root cert as a trusted root authority – which my external demo machine have). Specifically the Office clients have issues with not trusted certs.
The certificate is a SAN certificate covering sp.corp.local, trunk.corp.local and *.sp.corp.local. The certificate is installed on the SharePoint 2013 machine and on the UAG 2010 machine.
SharePoint with HTTPS
Before starting the UAG configuration let’s take a quick look at how the SharePoint 2013 farm is configured. In the SharePoint farm there is one Web Application using SSL on port 443. This Web Application is used to host Host Named Site Collections. This installation has two; sp.corp.local and teams.corp.local.
Install and update UAG 2010 to SP3
Installing Unified Access Gateway 2010 and patch it to Service Pack 3 is quite easy. But first of all – UAG 2010 SP3 does not support Windows Server 2012, so you have to dust off your old Windows Server 2008 R2 DVD’s and get used to that annoying Start button in the lower left corner.
Before inserting the DVD though, make sure that the OS is patched – yea, this machine should be a secure box and should have all the latest security hotfixes etc. Secondly make sure that you have the two required NIC’s wired in. Third – configure the NIC’s. If you’re uncertain on how to configure the NIC’s there is a good TechNet Wiki article called “Recommended Network Adapter Configuration for Forefront UAG” that is highly recommended as a starting point.
Once all that is set, insert the DVD and follow the wizard to install UAG. If your DVD is a UAG SP1 you need to apply the following patches, in this order, to get up on Service Pack 3.Once that is done a reboot is not out of order. Especially if you fiddled with the NIC’s and haven’t rebooted since.
Initial UAG configuration
Once you’re up on UAG Service Pack 3 it’s time to do the first time configuration of UAG. It’s a three step wizard that automatically starts when you fire up the UAG Management console the first time – and you can’t bypass it.
The first step is to connect UAG to your different NIC’s. As I said you need two of them – one for the internal network and one for the external network. In the Network Configuration Wizard (Step 1 in the UAG Getting Started Wizard) you have to choose which adapter to connect to the UAG Internal and External zone respectively. A good practice is to name your network adapters (in Windows) and give them describing names so they are not called just “Local Area Connection 1” and “Local Area Connection 2” – very easy to make stupid mistakes then…
The image below shows how I connected my “Corporate Network” NIC to the Internal UAG zone and the “External Network” to the External UAG zone. The machine have a third NIC – which I leave unassigned in this case.
Click Next when you’re done and finish the wizard. Step 2 in the Getting Started Wizard let’s you specify the UAG topology – leave it as as a single server and finish the wizard. Step 3 is the obligatory – do you want Windows Update and do you want to send usage data to Microsoft – I’ll leave it up to you to figure out the settings here.
Note: you can always get back to the Getting Started Wizard by choosing Admin > Getting Started in the UAG Management app.
Now all is set to start publishing some applications.
Creating a UAG Trunk
All applications in UAG are published in Trunks. A trunk consists of a portal and one or more applications. Each trunk have a set of shared settings between the different published applications – such as a public host name, certificate, URL Rules, endpoint policies, authentication and authorization settings etc.
Since we would like to publish an HTTPS SharePoint 2013 application we start with creating a new Trunk under the “HTTPS Connections” node in the UAG Management App.
This will start yet another wizard (YAW). In Step 1 choose to create a Portal trunk. In Step 2 we need to specify the name, public host name and IP of the trunk.
Step 3 is used to configure how the end-user will authenticate. Click Add to select (at least) one Authentication Server. If you don’t have any servers in the server list you need to add one. You have a lot of options here – but in most cases you will add an Active Directory server and connect it to your forest. You also need a service account for this – that needs permissions to retrieve user information and change password.
In Step 4 it’s time to select the external certificate. Hopefully you followed the instructions and have already registered the certificate on the UAG box. Choose the appropriate certificate and click next. If you’re like me and using a dummy domain (corp.local) you will get a warning here – it’s safe to ignore, we’re only writing a blog post.
The fifth step allows you to select what kind of endpoint protection you would like to use. Choose between UAG access policies or NAP. Choose UAG and continue to step 6, where you choose the endpoint policies – keep the default ones and continue. Then review your settings and click finish.
That’s it, you now have a trunk. All you now need to do is to save and then activate your new configuration. This is done through the cog wheel button in the management console.
You should now be able to test if you can access the trunk from a remote machine. The first time you access the trunk URL you will be asked to download and install the Forefront UAG client components. Depending on your configuration you will have limited access to your applications unless you install it. These components does a number of client security validations and they are also responsible for cleaning up your machine when you log out. Once logged in you should see an “empty” portal saying “No applications defined”.
If you don’t see anything like this or get errors here – it’s most likely related to certificates. Make sure that you have a correctly issued certificate registered with the Trunk. Also check the certificate chain and make sure that all the certs in the chain are trusted.
Add an application to the trunk
Now the time has come to publish our SharePoint 2013 Site Collection. In the Trunk you created under Applications click the Add button. YAW! In Step 1 you specify what kind of application you want to publish. You will find SharePoint 2013 under Web – it’s called “Microsoft Office(!) SharePoint Server 2013”.
Click next, give the application a name. In Step 3 leave all the endpoint policies as-is – you could easily spend a day fiddling with these if you have the time or need. In the fourth step choose to publish an Application Server. You should choose this if you have one server or a SharePoint farm behind some kind of load balancer. Step 5 is where you configure the actual SharePoint 2013 application. Add the name of your internal SharePoint 2013 site and specify the public host name. In my case my SharePoint 2013 site is published using HTTPS internally so I need to specify HTTPS/443 here – no SSL Termination.
Step 6 – Authentication. Choose an authentication server (use the same as you created and used for the Trunk). Here you also specify how rich clients authenticate with your application – in my case I choose to use both Rich Client integration and a Office Forms Based AuthN.
In the seventh step you configure how the application appears in the portal and in the eight and last step you can configure which users actually have access to your application, but just leave the default settings for now – that is everyone who can log in can also access the application (note, that this doesn’t mean they bypass any security in SharePoint).
Once you’re done with the wizard you should have two applications listed in your trunk – the portal and your newly added SharePoint site:
Now save and activate your new configuration.
Take it for a test-drive…
Close down your browsers, if you did test to log in to the Trunk earlier, and browse to the public portal URL once again. If you don’t see your application – log out of the portal and log in again.
You should see something like this:
Now, click on the application link and see how beautifully your SharePoint 2013 site is published through UAG. Test how you can drag and drop documents, open them in Office clients, edit lists in Quick Edit mode and work just perfectly!
Publishing the second SharePoint 2013 Site Collection is exactly the same – you add a new Application to the trunk and follow the wizard.
Summary
In this post I showed you how to install, patch and configure Forefront Unified Access Gateway (UAG) 2010 Service Pack 3 to be able to publish SharePoint 2013 Web Applications and Site Collections. It’s a pretty straightforward approach but involves some crucial steps. If you previously published SharePoint 2010 applications using UAG you noticed that there is no difference with regards to setting it up. The difference are in the details, specifically in the URL rules applied and a new set of policies.
-
SharePoint 2013: SharePoint Health Score and Throttling deep dive
The SharePoint Health Score was introduced in SharePoint 2010 and plays an even more important role in SharePoint 2013. The Health Score determines the health of a SharePoint server/web application on a scale from 0 to 10, where 0 is the most healthy state. SharePoint automatically starts throttling request once the Health Score is to high. The Health Score is can be calculated using many parameters, such as memory usage, concurrent requests etc. In this post I will give you some details on how the Health Score works, how you can troubleshoot it and how you can use it and how you can configure it.
Note: this article contains some registry changes that never ever should be done on a production server, unless told by Microsoft support...or whatever...
What is the Health Score?
Let’s start with some background facts. Stop whining, I know plenty of peeps have already blogged about it but I think it is good to have all the facts in this same blog post.
The Health Score is as an integer between 0 and 10, where 0 represents a really good health of the server, or rather Web Application, and 10 is where you do not want to be. SharePoint continuously calculates the Health Score and for each and every request the Health Score is sent as an HTTP Response Header, called
X-SharePointHealthScore.This Health Score is used by (or at least should be used by) client applications to determine the health of the web application and then use it to make a decision on whether it should stop hammering the web application or not. If the Health Score reaches up to 10, SharePoint itself will start to throttle requests (which we’ll talk about in just a bit).
Worth noticing here is even though this article is written for SharePoint 2013, most of it is valid on SharePoint 2010 as well.
How the SharePoint Health Score is calculated
So, how does SharePoint calculate this Health Score? When a SharePoint Web Application spins up, internally a new thread is created (you can see it in debuggers with the name SPPerformanceInspector). This thread regularly reads performance counters and calculates the Health Score. Each time it calculates the Health Score it will log this to the Trace Logs (Category=Http Throttling, Level=Verbose):
“The current health score for web application is X”
Performance Counters
The Health Score is calculated from a set of Performance Counters. By default SharePoint 2013 (and SharePoint 2010) uses two performance counters for this:
- Memory/Available MBytes
- ASP.NET/Requests Current
You can easily retrieve these performance counters by using the following cmdlet:
Get-SPWebApplicationHttpThrottlingMonitor http://app.contoso.com
You can also use the
Set-SPWebApplicationHttpThrottlingMonitorcmdlet to modify the current monitors.Buckets, what is that!?
When running the cmdlet above, to retrieve the performance counters used to calculate the Health Score, you should have noticed that each associated performance counter has a property called
AssociatedHealthScoreCalculator.This is an array of values, or a set of Buckets, that SharePoint uses to calculate the Health Score. There are 10 buckets in each of those (you can have fewer). Each bucket represents a Health Score value and corresponds to a lower limit of the performance counter. For instance if you have 24 current ASP.NET requests – you will have a Health Score of 4 and if you have less than 200 MB of RAM available you will have a Health Score of 6. The Health Score of the Web Application is equal to the largest Health Score of all measured performance counters.
Adding your own ,monitor
To add your own performance counters to monitor you need to use the API. Here’s a sample on how to add the CPU usage as a monitored object. You should not add this one into any production environment, but it can be used in testing scenarios when you would like to see what happens when the Health Score is high.
$throttle = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings $throttle.AddPerformanceMonitor("Processor", "% Processor Time", "_Total", 10, $true) $throttle.Update()This will create a new monitor for the Web Application that checks the processor time on the machine. It will only have one bucket. So if the processor time is larger than 10 it will yield a Health Score of 10.
Refresh Interval
The Health Scores are by default updated every 5 seconds. You can get (or set) this setting by looking at the
HttpThrottlingSettingsof a Web Application like this:$throttle = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings $throttle.RefreshInterval
To avoid having spikes in performance counters creating high Health Scores, the actual value is calculated from an average of a number of samples, by default 12 samples. This can be configured using the
NumberOfSamplesproperty on theHttpThrottleSettings.How SharePoint uses the Health Score
Now you know a little bit more on how the SharePoint Health Score is calculated. So what is it used for then? SharePoint internally uses it for mainly two reasons; HTTP throttling and for the Request Management Service.
HTTP Throttling
Whenever the SharePoint Health Score reaches the value of 10, SharePoint 2013 starts its throttling mechanism to protect the servers. As soon as the Health Score is below 10, the servers stops the throttling. The actual throttling is divided into two steps – the first and second stage. After receiving a Health Score of 10 and SharePoint enters throttling it will start with the first stage and requests will start to be throttled. After one minute of throttling the throttling enters the second stage and starts throttling even more. What’s being throttled in the different stages depends on a set of throttling classifiers. Out of the box (in SharePoint 2013) there is only one classifier used – which checks if the current request is a search crawler. This classifier starts throttling request directly in the first stage – all other requests are handled as usual. When the throttling enter the second stage all requests will be throttled (default, if you have any custom classifiers some requests might be allowed).
Throttled requests are met with the Error message above: “The Server is busy now. Try again later”, and the HTTP Response: “503 Service Unavailable”.
When entering the second stage of throttling, you will see the following lines in the ULS logs. The logs also includes the “Excessive” counters.
To the Server is Busy response SharePoint also adds two Response Headers; one called “Retry-After” which contains a numeric value which should be taken as an indication (by clients) when a retry operation should be done (it’s always has the same value as the Health Score refresh interval) , and a second one “SharePointError” which always has the value of “2”.
By the way, if you notice the nice little error icon in the Busy error. That image is not throttled, why? Well, I can tell you…there’s actually an exclusion for that specific image with regards to resource throttling. Requests to that image will never be throttled.
HTTP Throttling of POST Requests?
There’s this myth out there that POST and PUT requests are never throttled. Well, that is to good to be true! My tests and reverse engineering skills shows that all requests are throttled in SharePoint 2013 (I would be glad to be proven wrong :). Fortunately Office 2010 and later has the “Upload Center” which will retry any save operations and eventually succeed once throttling has stopped.
But, using one of the built-in classifiers we can actually make sure that our precious documents are saved regardless of HTTP Throttling, and this is how this is achieved.
$classifier = New-Object Microsoft.SharePoint.Utilities.SPHttpUserAgentAndMethodClassifier -Args "","POST","HttpMethodMatch", "Never" $throttle = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings $throttle.AddThrottleClassifier($classifier) $throttle.Update()First of all we create a new classifier of the type
SPHttpUserAgentAndMethodClassifier, when creating that object we pass in “POST” as HTTP method, make sure it’s using the HttpMethodMatch method to match the method :), and finally we tell it to never throttle these requests. We add this classifier object to the throttle settings of the web application and badabing, POST operations are not longer throttled.Turning off HTTP Throttling
Ok, so you’re fed up with this throttling and want to turn it off. It’s an easy operation and you can do it from Central Administration or from PowerShell. In PowerShell just update the PerformThrottleproperty of theHttpThrottleSettingsof the web application. And in Central Administration you select the Web Application for which you want to turn off throttling, then select General Settings > Resource Throttling, then at the bottom of the dialog you will find HTTP Request Monitoring and Throttling.HTTP Throttling and automagic garbage collection
The HTTP Throttling is actually a quite smart implementation, and internally tries to fix your machine if throttling occurs. One thing that is important here is if you start fiddling with the different performance counters used for calculating the Health Score, make sure that you do not remove the monitor that monitors memory usage. That plays a special role in the throttling scenario. If that specific monitor is considered an “Excessive” monitor, which means that it has an Health Score of 10, it will actually force a Garbage Collection.
Request Management
SharePoint 2013 introduced the new Request Management Service, which is used to “redirect” requests to specific servers based on rules and/or health. When using health based rules, it is the SharePoint Health Score that is used to determine the health of a server. Whenever a machine receives a request it will update the Health Score in the Request Management service, which makes the Health Based Request Management a very efficient way to “load balance” the requests for optimal performance.
For more information about Request Management read Spencer Harbars series about it.
How to use the SharePoint Health Score
Whenever you’re building a client that interacts with SharePoint 2013, for instance a SharePoint App, a Windows 8 app or a Windows Phone app, then you should strongly consider always take the Health Score in consideration. You will receive the
X-SharePointHealthScoreresponse header for each and every CSOM, REST or web service call to SharePoint. If the value is 10 (and SharePoint starts throttling) then you will most likely not be able to get data. If your application is constantly polling SharePoint for information, perhaps you should increase the delays between your calls when the Health Score raises above a predetermined number.Andrew Connell has a small code sample on how to leverage this in a Silverlight app.
The SPPerformanceInspector
You could also use the Health Score in traditional SharePoint Farm solutions (timer jobs, or other long running or resource consuming operations) using the
SPPerformanceInspectorobject. Here’s a quick sample how you can show the data in a Web Part:SPPerformanceInspector pi = SPPerformanceInspector.GetInspector(SPContext.Current.Site.WebApplication); this.Controls.Add(new LiteralControl( String.Format("Current Health Score: {0}<br/>", pi.HealthScore))); this.Controls.Add(new LiteralControl( String.Format("Is throttling: {0}", pi.IsInThrottling())));This Web Part retrieves the performance inspector object and then prints out the current Health Score for the current Web Application on the current machine. Also this Web Part will print out if throttling is currently going on (which you never should see anyways, since you’re throttled!).
How to debug the SharePoint Health Score
To wrap this little post up I will give you two great debugging tips – they both involve editing the registry (you’re warned!). First of all if you’re building a remote application that communicates with SharePoint and you want to test how it behaves at different Health Scores. Then there’s a neat trick to give a server a constant Health Score value. You do this by adding a new DWORD key called
ServerHealthScoreto theHKLM\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\15.0\WSShive and then do an IISRESET. You can give this key any value from 0 to 10 and the Response Header will always show that value. It will not invoke throttling though.It will of course log a warning in the ULS telling you how bad this really is:
The second tip is a cool debugging feature, and its also a small regkey hack. In the same hive as above create a new DWORD key called
DebugThrottleand give it the value of 1.Once again you need to do an IISRESET. After that you can browse to any page and then append
debugthrottleto the address, for instance: http://site.contoso.com/debugthrottle. This will give you a sweet looking dashboard with all the details you need about the health of the server/web application. You can see the monitors and their Health Score history, the throttling stage, your classifiers etc.Summary
There you have it! All you need to know about the SharePoint Health Score and HTTP Throttling. As you can see it’s a really nice feature if used correctly – by you, by third party vendors (I’m pretty sure 99% of the vendors do not care about this though) and by SharePoint itself. Just one final word of warning – if you’re fiddling with monitors and classifiers make sure that you thoroughly test them, it’s really easy to get things funked up here…
-
Office Web Apps 2013: Securing your WAC farm
With this new wave of SharePoint, the Office Web Apps Server (WAC – I don’t like the OWA acronym, that’s something else in my opinion) is its own server product, implementing the WOPI client protocol, which allows a client to retrieve documents from SharePoint on the behalf of the user. Documents will flow from the WOPI servers (SharePoint, Lync, Exchange etc.) to the Office Web Apps Server – this means that potentially confidential information will be transferred from the SharePoint environment and stored/cached on another server. This could result in unnecessary information leakage and compromise the enterprise security.
In this post I will walk through a number of steps that you can do to properly secure your Office Web Apps 2013 farms. And you should seriously consider and implement most of these methods.
Note: this post focuses on the Office Web Apps Server and not a WOPI client in general (but if you’re building your own you should consider security as well!).
The WOPI protocol specification and security
Note: I will not cover how WOPI clients and servers implements the server to server authentication and authorization.
WAC runs as Local System
To start with it is very important to know that the Office Web Apps Server 2013 runs as the Local System and Network Service on the machine it is installed on. There is no service account or anything! This means that you cannot protect your systems using dedicated accounts etc., like you do with SharePoint, SQL and other applications.
The images below shows the Office Web Apps Windows Service, which runs as LocalSystem.
And this image shows some of the applications pools in the IIS on an Office Web Apps machine.
The advantage of using these local accounts is that it makes installation and configuration easier. But it is very important that you are aware of this configuration.
SSL is a requirement!
Exposing the Office Web Apps Server over HTTPS should be a requirement in my opinion. There is no reason not to. Having it on HTTP will only cause trouble for you; for instance if your SharePoint uses https you will not be able to render the iFrame containing the document (aka WOPI Frame) since you’re not allowed to show http content in an https environment. But first and foremost you’re sending data in clear text.
So what about SharePoint on HTTP then? Well, if you’re using SharePoint 2013 you should seriously consider running that over HTTPS as well – that IS a best practice. SharePoint 2013 leverages several technologies that sends tokens and credentials over the wire, OAuth for instance, so in order to have a secure environment make sure you use HTTPS for SharePoint as well. If you are running SharePoint on HTTP you must fiddle with the security settings in SharePoint to allow OAuth over HTTP – and this is not a good thing.
Certificates are king!
Any WAC farm running on SSL must have a certificate for the HTTPS endpoint. You can use self-signed, issue certificates using a Domain CA or buy a certificate. When you’re creating the WAC farm, using New-OfficeWebAppsFarm, you can/should specify the certificate.
For any SharePoint, WAC and even SQL installations nowadays certificates are more and more important. If you’re on the verge of deploying these in your organization you should consider deploying a Domain CA – which is not a lightweight task.
Securing the communication using IPSec
If you for some reason do not run HTTPS on SharePoint and/or WAC you could consider implementing IPSec. Unfortunately there is no button in the Control Panel that says “Use IPSec”. This is something that requires careful planning and testing. So going SSL might be an easier way. But consider the scenario where you have an internet facing web site which leverages WAC and using the HTTP protocol – then you should consider using IPSec for the communication between SharePoint and Office Web Apps Server.
Firewall considerations and requirements
When setting up your Office Web Apps Farm you should also configure the firewall for the WAC machines. Office Web Apps uses four different ports. It uses 80 and 443 for HTTP and HTTPS, that’s used by the end-users and the WOPI Server/Client communication. Internally Office Web Apps uses port 809 (HTTP) and 810(HTTPS) for communication between the WAC machines. I’ve only seen 809 in use, which is the default. There is no way (I’ve found at least, but internally WAC has a switch to use port 810) to configure WAC to use port 810 and if you do find a way, it’s likely unsupported. The things sent over the wire using the admin channel (809) is mainly health and configuration information for the WAC farm, but it would be nice to be able to secure this channel as well (IPSec!).
When installing WAC the Windows firewall is configured to allow incoming TCP connections on port 80, 443 and 809.
As always it is a good practice to evaluate these default rules and if you’re not using port 80, disable that port. For port 809 it might also be a good practice to make sure that it only allows incoming connections if they are secure (i.e. implement IPsec).
Preventing rogue machines
So far we’ve been talking about how to secure information being transmitted from and to the Office Web Apps farm. Let’s take a look at Office Web Apps farm security from another angle. Joining a new WAC machine to an Office Web Apps Farm can be quite easy. The only thing that you need is local administrator access on the WAC machine that is the master (the
Get-OfficeWebAppsMachinegives you the master machine). Depending on how you’re having your (virtual) metal hosted this might be a problem, too many sysadmins have to much permissions out there. If you have this access then you can easily join a rogue machine to the WAC farm and take control over it, without the users/client knowing anything about it.There are a couple of methods you can and should use to protect the WAC farm. And the error messages below can also be a good troubleshooting reference…
Master Machine Local Administrator
If the account trying to create the new WAC machine does not have local admin access on the machine specified when joining the WAC farm you will simply get an “Access is denied”.
As a side note; if you’re not running the cmdlet using elevated privileges you will get an “You must be authenticated as a machine administrator in order to manage Office Web Apps Server”.
Using the Firewall
I already mentioned the firewall. If the machine joining the WAC farm cannot access the HTTP 809 channel the New-OfficeWebAppsMachine will throw a “The destination is unreachable error”.
This is a fairly easy way to protect the farm, but if the user has local admin access on the master machine it can easily be circumvented.
Certificate permissions
If you’re using a domain CA, make sure that you protect the private key using ACL’s, or if you’re buying a certificate make sure to store the certificate private key in a secure location. If you’ve specified a certificate when the Office Web Apps farm was created, which you should have, then the user cannot join the new machine – regardless of local machine admin, since the user cannot install the certificate locally. The error message shown is “Office Web Apps was unable to find the specified certificate”.
Using an Organizational Unit in Active Directory
The way that Microsoft recommends to secure your WAC farm is to have a dedicated OU in Active Directory where the computer accounts for the WAC farm is located. When joining a new machine to the farm the cmdlet verifies that the account is in the OU specified by the WAC configuration. If it’s not then you’ll see the “The current machine is not a member of the FarmOU”.
The Farm OU is specified when creating a new WAC farm or using the
Set-OfficeWebAppsFarm/cmdlet. The only caveat with this OU is that it has to be a top level OU in Active Directory. Creating that OU in your or your customers AD might cause some headache, but if you want to use theFarmOUas protection method for your farm it has to be this way. That’s the way it is designed!Also having all the WAC servers in a OU gives you other benefits; such as using Group Policies to control the WAC servers.
Limit the WOPI Server and host access
Now we’ve seen how we protect the farm from rogue machines and data tampering. Another issue with the WAC farm in it’s default configuration is that any WOPI Server can use it. Might not be a big problem for most of the internal installations, but what if you’ve designed a WAC farm and someone with a huge SharePoint collaboration implementation connects to your WAC farm. It can sure bring it down. Or if you’re exposing your Office Web Apps farm on the internet anyone on the internet can potentially use it.
For this purpose there’s a cmdlet called
New-OfficeWebAppsHost. This cmdlet allows you to specify host names that will be accepted by the WAC farm. The cmdlet interprets any domain with a wildcard. For instance the following cmdlet will allow all WOPI Servers on contoso.com (www.contoso.com, extranet.contoso.com etc.) to contact the WAC farm:Set-OfficeWebAppsHost -Domain "contoso.com"
Do not forget to do this!!
Summary
You’ve seen quite a few ways how to protect your WAC farm from information leakage, rogue machines, undesired excessive usage etc. Using HTTPS and certificates together with a dedicated OU in Active Directory will give you the most secured WAC Farm. Hopefully you also understand a bit more on how Office Web Apps Server works internally. It’s a magnificent and simple server product, but it should be handled with care.
-
Sharing a Workflow Manager 1.0 farm between multiple SharePoint 2013 farms
SharePoint 2013 introduces a whole set of new and improved features. One of the things that is both new and vastly improved is the Workflow platform. Workflow is no longer a part of the SharePoint core infrastructure, but instead a separate server product. Even though ye olde Workflow platform, 2010 style, is still in the product for backwards compatibility. SharePoint 2013 leverages the Azure service called Workflow Manager 1.0. (Not the cloud version but a local on-premises installation).
The Workflow Manager 1.0 server application should be installed on a separate set of servers – not on the SharePoint servers (remember; just because you can doesn’t mean you should). This introduces for some organizations more investments in physical and virtual hardware. So, what happens if you have several farms? Do you need several Workflow farms? That can be expensive! Well, as always in the SharePoint world - It depends! If you come to the conclusion that you could use the same Workflow Farm and hardware for managing all your SharePoint Farms workflows then you can actually use the same Workflow Manager 1.0 farm. Usual disclaimers apply here – it’s all up to you and your organization to decide upon designing your workflow infrastructure for this scenario, remember this can make backup/restore and DR a bit problematic since now your farms are sharing the same workflow databases etc.
Let me show you a multi-tenant Workflow farm, how it works and how you do to set it up properly…

Multi-tenancy in Workflow Manager 1.0
Azure Workflow Manager 1.0 is multi-tenant aware through Scopes. “A scope is a named and securable container for Activities, Workflows, Instances, configuration and child Scopes.” – that is directly stolen from the Workflow Manager documentation. Everything that runs inside Workflow manager belongs to a Scope. By default a Root Scope is created when you set up the Workflow Manager farm. You can see details about the Root Scope by navigating to the HTTP/HTTPS port of your farm. The URL should look something like this: http://wffarm.corp.local:12291 for HTTP and https://wffarm.corp.local:12290 for HTTPS.
Connecting SharePoint 2013 to the Workflow Manager 1.0 farm
I’m not going to dive in to how to configure the Workflow Manager farm; it’s a wizard or PowerShell thingie and it’s very well documented – you can’t fail! Once you have the Workflow farm up and running you MUST install the Workflow Client 1.0 components on all the Web Server in your SharePoint 2013 farm (you can get the bits using the Web Platform Installer).
All you have to do then is to register the Workflow Manager 1.0 farm with the SharePoint 2013 farm using the Register-SPWorkflowService cmdlet. This will register the SharePoint 2013 farm with the Workflow Manager and vice versa and it will also create the Workflow Service Application Proxy in SharePoint.
You execute the cmdlet like below and pass in a URL to a site in your SharePoint 2013 farm (any site will do) and the URL to your Workflow Manager farm (if you’re using HTTP you also need to add the AllowOAuthHttp parameter).
Register-SPWorkflowService -SPSite https://teams.corp.local -WorkflowHostUri https://wffarm.corp.local:12290Not only will this cmdlet do the things mentioned above, it will also create a new Scope in the Workflow Manager farm (under the Root Scope) called SharePoint. You can verify this by navigating to Workflow Manager URL and append /SharePoint, like this: https://wffarm.corp.local:12290/SharePoint. In the XML returned you can see the URL to the SharePoint 2013 metadata endpoint (https://teams.corp.local/_layouts/15/metadata/json/1) under the SecurityConfigurations element. Now, that was one SharePoint 2013 farm connected to the Workflow Manager farm.
So, let’s jump on over to our second SharePoint 2013 farm, that we now decided should use the same Workflow farm! If we run the same cmdlet in that farm (using a different SPSite parameter URL of course) we get a nice error that says “An existing scope named “SharePoint” already exists in the workflow server”.
You see, this scope is already taken by some other farm (the one we previously connected)! The clever one might think that oh, I’ll just use the Force parameter of that cmdlet and override this. Well, if you’re that smart then you will hijack that Scope from the other farm. The Scope will be recreated and the other farm will have a non-functional Workflow Service (the security configuration will now point to the SharePoint farm that hijacked the Scope).
Note: the Force flag might be useful if you really want to re-create the Workflow connection using the same Scope name.
Instead you should use the ScopeName parameter of the Register-SPWorkflowService cmdlet. That parameter will create a new Scope in the Workflow farm, and with that create an isolated container for this new SharePoint farm. So on our second farm we run this PowerShell cmdlet:
Register-SPWorkflowService -SPSite https://farmb.corp.local -WorkflowHostUri https://wffarm.corp.local:12290 -ScopeName FarmBOnce this command is issued your Workflow Manager farm will have two child Scopes under the Root Scope; SharePoint (the first farm we connected) and FarmB (the second one with the ScopeName parameter). You can verify the second scope by browsing the the Workflow farm and append the Scope name to the URL, like this: https://wffarm.corp.local:12290/FarmB.
Now you have two SharePoint 2013 farms using the same Workflow Manger 1.0 farm, and each SharePoint farms workflows are isolated using the Scopes in Workflow Manager. Cool!
Summary
I’ve just shown you how easy it actually is to share a Workflow Manager 1.0 farm between several instances of SharePoint 2013 farms. Now it’s up to you to decide on weather this is an appropriate way for your organization to build your infrastructure or not. Remember to plan your resources and your DR strategy!
-
Summing up the year of 2012 and embracing 2013
Only one day and a few hours left of the year of 2012 and the time has come for me to make my (now traditional) summary post. I’ve done it now for six years (2011, 2010, 2009, 2008, 2007 and 2006) and it’s always fun to look back at what has happened during the last 12 months.
This year has been an interesting year to me and contained some really interesting milestones, happenings and events.
Writing
This year I’ve written basically the same amount of posts as in 2011, even though a new SharePoint version has been released. Basically it comes down to that I’ve been working a lot and haven’t had time to finish up some of the posts I’ve started and I’ve been busy with other commitments in the community. Anyways, I hope you enjoyed what I’ve written – I try to keep the quality up rather than the quantity.
What’s really cool is that I now on average have more that 1.300 daily subscribers to my blog, a 30% increase since last year. Thank you!
The most popular posts this year has been:
- How Claims encoding works in SharePoint 2010 – a post you should read IMO and still valid for SharePoint 2013
- Visual Guide to Azure Access Control Services authentication with SharePoint 2010 – part 1 – The first part of six posts in the series.
- SharePoint 2013 – Introduction to the Minimal Download Strategy – MDS – An MDS overview.
- SharePoint 2013 – Claims is the new black – the post title says it all
- SharePoint 2013 – A look at the hardware and software and other requirements – my pov on the hw and sw requirements
The most popular post (from all years) is still the “Fix the SharePoint DCOM 10016 error on Windows Server 2008 R2” post.
No books this year – done that once…
MVP again..
I was re-awarded the Microsoft Most Valuable Professional, MVP, award for SharePoint for the third consecutive year. I’m honored by still being a part of this group of community contributors.
Microsoft Certified Architect – MCA
What I’m most proud of this year is that I passed the Microsoft Certified Architect for SharePoint, MCA, certification. This is a certification, currently only held by five individuals worldwide, which proves that you can “architect” a business and technical solution based on SharePoint and related technologies. What I do like about this certification is that it is not just a test or a lab (you get those tests while doing the MCM/MCSM, which is a pre-requisite for the MCA) but instead it is a mix of interview, business case, portfolio, presentations and Q&A’s, where you really must show that you understand business requirements, budgets, time plans etc. You can read more about my take on the program here – What is a Microsoft Certified Architect?
Conferences and travels
I got my fair share of travelling this year as well, a bit less than last year – which my family appreciated though. I had the opportunity to speak at a couple of conferences – where the International SharePoint Conference 2012 was the highlight. This conference was something extraordinary for both the attendees and us speakers. I really enjoyed working together with the team on the dev-track building out our solution and sessions and it was so fun having the whole team on the front row supporting (and a little bit of heckling) each other during the three days. I will be back next year in London!
I also had a blast at the first SharePoint conference in Croatia, and I hope I’m invited again and as usual our local SharePoint and Exchange Forum – which just keeps growing!
Microsoft arranged the SharePoint Conference 2012 in Las Vegas to unveil the spanking brand new SharePoint version…well it turned out that the product was released a month ahead, so the conference didn’t have much new information. In my opinion this was a pretty bad conference – the depths of the sessions was to low, to many of the speakers should not have been allowed on stage, and what a total disaster when all Microsoft talked about was how good the cloud is and they had no Internet connectivity. I really hope that the conference team get their stuff together and rethink a lot of things for the next conference!
The Cloud
I can’t really write a summary post of 2012 without talking about the Cloud. Microsoft (and other vendors) really put all their money on the cloud this year – nothing new with this, it’s been going on for years, but this year it’s more clear than ever. Well, the cloud is nothing new – it’s just a new name for the Internet, Application Service Providers, etc etc, it’s more of a marketing term.
Since the release of Office 365 the transition of SharePoint to the cloud has started (BPOS was first but SharePoint wasn’t cloud ready at all at that time). With this new wave of SharePoint and 365 the SharePoint cloud offering is even more evident – and we don’t know now where it will end, I’m not sure even Microsoft knows that yet.
We’ve been running 365 for a year and a half now and we still suffer a lot from different strange issues (that could be a post on it’s own!). Hopefully once upgraded the service will be more stable and useful.I might sound a bit doubtful about the cloud – and I am. For the vast majority the cloud (read 365) is a great option instead of hosting and managing their own instances. But for large enterprises (the clients I’m normally working with) 365 is not even an option…
Only time will tell…
Predictions
This is the part I really enjoy writing and thinking about at this time of the year – looking back at my last years predictions and looking in the crystal ball for the upcoming year.
Last years predictions wasn’t that good – Silverlight is till not dead (someone hooked it up to a CPR machine) and all the browser vendors seems to be on some kind of honeymoon. My wish that 2012 was a bit less cloudy failed miserably! The only thing I think I got right was that Windows Phone would have momentum and by looking at some stats there’s some truth in that.
So what do I think about next year?
- Cloud, cloud and cloud! The cloud marketing will continue, it will be shoved down our throats, we’ll be so sick of it by the end of 2013. But this will be the way forward. It will be really interesting to see what will happen to traditional “on-premises” products – will they vanish? Just take a look at how Microsoft has discontinued a lot of products this year!
- Identities, certificates, federation! I think one of the most important things for next year and the years following are having a good identity infrastructure and identity management in place. This should be a priority for all companies and architects working in our business. It’s also the basic requirement to get any cloud services to work properly.
- Apps, devices and integration! For the cloud, hosted services etc to work the integration story must be better. We’ve seen a good start with Windows 8 (RT), Windows Phone 8, Office 2013 etc – but it’s not fully there yet. I think that this is what will be improved over the next 12 months and that might also be the key differentiator between the three large “ecosystems” – Windows, Linux/Android and the fruit camp.
- SharePoint community! The SharePoint community has grown really large and it has reached some kind of “critical mass”. I’m not saying it’s to big or anything but I see the community being more divided than ever splitting up in different directions with different specializations, aspects, mind-sets etc. The number of conferences, user group is ever increasing. This year the community will change (due/thanks to the cloud) – we will see a big change in focus, we will see SharePoint professionals being Azure, Windows 8/Phone, JavaScript professionals.
What do you think?
Thank you and a happy new year!
With that I would like to wish you all a Happy New Year and say thank you for 2012. I‘ve been having my doubts about 2013, but now I ‘m really looking forward to it. I have some cool events and travels planned, I know that my work at Connecta will contain some interesting opportunities and I know that the demand for SharePoint is ever increasing…

See you on the other side!
-
SharePoint 2013: Unable to install the pre-requisites on a disconnected machine
You all know that you need to install the SharePoint 2013 pre-requisites before installing SharePoint 2013 – this is done either online or offline using the pre-req installer. All the requirements are listed in the Hardware and Software requirements for SharePoint 2013 Technet article.
Once in a while you need to do the installation on a disconnected machine, that is a machine that is not connected to the interwebz. Then you typically Bing for a fancy script that downloads all the pre-reqs for you and you run the pre-reqs installer in unattended mode. That is where you pass in the local path to all the downloads either through a file or through the command line (I’m not going to cover that – it’s all over the web and even in the TechNet article mentioned before).
Trouble in paradise?
One thing that you might stumble upon when doing a disconnected install of the SharePoint 2013 pre-reqs is that your scripts or the pre-req installer just keeps asking you to run the installer again and it just says that you need to restart your server and run the installer again. After doing that for a couple of times you most likely start inspecting the log files and you will see stuff like this:
2012-12-03 21:38:40 - Installing Microsoft WCF Data Services 5.0 2012-12-03 21:38:40 - "\\server\SP2013\PrerequisiteInstallerFiles\WcfDataServices.exe" /quiet 2012-12-03 21:42:50 - Install process returned (-2146762486) 2012-12-03 21:42:50 - [In HRESULT format] (-2146762486) 2012-12-03 21:42:50 - Last return code (-2146762486) 2012-12-03 21:42:50 - Install needs restart
We’ll it looks like it needs a restart – but what about that HRESULT code? Let’s turn it into a hexadecimal representation – 0x800B010A. Ahaaa! Ya’ll know your HRESULTs, right. It starts with 0x800B01?? so it must be some certificate issue, and it sure is.
As you can see it fails when installing the WCF Data Services 5.0. Let’s run that installer by itself. When we do that we’ll see an error that clearly says that it is a certificate error – A certificate chain could not be built to a trusted root authority.The installer log file for WCF Data Services also tells it right in our face:
[0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to verify certificate chain policy status. [0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to get certificate chain for authenticode certificate. [0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to verify signature of payload: wcf_dataservices
So there’s something wrong with our certificate validation chain here…
These darn certificates!
Having disconnected machines means one or two things with regards to patching – either they are not patched at all or the admins control the patching themselves and don’t let the right ones through. The problem here is that the machine experiencing this problem does not have its root certificates updated. It’s an easy fix – make sure you let those updates from Windows Update through or update the root certificates manually.
A manual update, which should only be your last resort, can be done by downloading the Update for Root Certificates for Windows XP [April 2013] (KB931125). Yup, they are for Windows XP but works perfectly on a Windows Server 2008 R2 (that’s where I tested it). Download the rootsupd.exe file and run it on your server before the running the pre-req installer and all should be running fine from now on.
Summary
A short explanation and solution for something that hopefully isn’t that common – since all of you have patched servers right?
-
SharePoint 2013: Building your own WOPI Client, part 4, now Search enabled
Well, I thought I should write another episode of my Building your own WOPI Client series, here’s the links to the previous episodes part 1, part 2 and part 3. This time around we’ll dig into another of the different actions that a WOPI Client can surface – the interactivepreview mode.
Background
As you’ve seen in the previous posts we can build a viewer and an editor for C# files, to be used in document libraries for instance. What if we would like to lift up and enhance our custom file formats in search, just like Office Web Apps Server does with the Office files. We’ll you can do that very easy, and you should! In this post I’ll show you how to surface the preview mode in a Search flyout. This also means that we’re going to take a look at the new SharePoint 2013 Search Engine, the Design Manager and some funky HTML syntax.
Configure indexing for C# files
The first thing we need to do is to actually make SharePoint indexing our C# (.cs files). This is done in two steps. First we need to tell SharePoint 2013 Search to include the .cs file extension and then we need to tell the search engine what filter to use. To include .cs files in the index there is two ways to do it, using PowerShell or using the Central Administration. In Central Administration you go to the Search Service Application and choose File Types, then click on New File Type and type in cs. That’s it. Using PowerShell the following snippet works (as long as you only have one SSA):
$ssa = Get-SPServiceApplication | ?{$_.TypeName -eq "Search Service Application"} New-SPEnterpriseSearchCrawlExtension -name cs -SearchApplication $ssaOnce that is done we need to fiddle a bit with the registry (usual regedit disclaimers apply here). Start regedit.exe and navigate to HKLM\SOFTWARE\Microsoft\Office Server\15.0\Search\Setup\ContentIndexCommon\Filters\Extensions and choose to add a new Key. Give the key the name “.cs” (without quotations) and then set the default value to the following (fancy crafted) Guid: {0FF1CE15-002C-0000-0000-000000000000}. All you now have to do is to restart the Search Engine like using the PowerShell below and then start a full crawl:
Restart-Service OSearch15
Now when you search for any content in your .cs files it should look something like this:
It’s something, but not what we want…
Building the WOPI Preview
Before we start customizing the UI let’s build the interactiepreview mode of the WOPI client. Just as we did in the previous post, when we added the edit mode, we need to add a new action to the discovery file and then remove and add the bindings again (for that PowerShell, see previous post). This is the action I’ve added to my WOPI Client:
<action name="interactivepreview" ext="cs" default="false" urlsrc="http://wacky.corp.local/csharp/preview.aspx?<ui=UI_LLCC&><rs=DC_LLCC&>" />As you can see that the urlsrc points to a preview.aspx web form. That web form is essentially a copy of the viewer.aspx, with the exception of the menu which I have removed. In a real world scenario you might consider having very different viewers for the normal view mode and the interactivepreview mode. You want this process to be fast and you might just generate a thumbnail or something.
When all is compiled and deployed, and you have removed and re-added the WOPI bindings, it’s time to hook this up to search.
Creating the Display Templates
Let’s build the search interface. The SharePoint 2013 Search Interface does not any longer require extensive XSLT skills but instead plain ol’ HTML hacking. Every Result Type in SharePoint 2013 Search can have it’s own Display Templates, and there’s two of them; the normal item view and the flyout/hover view. The easiest way to create new Display Templates is to copy an existing one. For this sample let’s use the Word Display Templates.You will find the Display Templates in the new Design Manager (which exists in a Search Center or a site with the Publishing features enabled). You access the Design Manager through the Site Settings menu. Once in the Design Manager you should choose Display Templates in the menu on the left hand side. Then I usually filter on Target Control Type, to narrow down on the Display Templates used for Search; choose SearchResults and SearchHoverPanel. Now scroll down to the Word Item and Word Hover Panel. You’ll see two of each but we’re only interested in the HTML files for now. Download those two files to your local disc and name them Item_Cs.html and Item_Cs_HoverPanel.html respectively.
Now open these two files in your favorite HTML tool – for me that is Visual Studio 2012. There’s a lot of weird stuff in here as you will notice, lots of comments and JavaScripts. I’m not digging into all the details here, that’s for Randy Drisgill or someone else to do.
Let’s do some simple string replacement. In both files replace “Word” with “Cs” as the first thing. Next update the mso:MasterPageDescription to something more appropriate and the title tag, perhaps replace “Microsoft Word” with “C#”. In the Item_Cs.html file locate the second div having an id equal to "_#= $htmlEncode(itemId) =#_" and remove everything inside it and replace with this instead:
<div class="ms-srch-item-title"> <h3 class="ms-srch-ellipsis"> <a class="ms-srch-item-link" href="_#= ctx.CurrentItem.Path =#_">_#= ctx.CurrentItem.Title =#_</a> </h3> </div> <div class="ms-srch-item-body"> _#= Srch.U.processHHXML(ctx.CurrentItem.HitHighlightedSummary) =#_ </div> <div id="_#= $htmlEncode(hoverId) =#_" class="ms-srch-hover-outerContainer"></div>What this does is that we’re first rendering the title of the document with the value ctx.CurrentItem.Title. All these JavaScript variables are enclosed in __#= … =#_. Then we output the hit highlighted summary using a built-in method (Srch.U.processHHXML). Finally there’s an empty div which will contain the flyout (The flyout is defined in the Item_Cs_HoverPanel.html file). There is one final thing that we need to do and that is to update the hoverUrl JavaScript variable in this Item_Cs.html file. It should point to the Item_Cs_HoverPanel.js file like this:
var hoverUrl = "~sitecollection/_catalogs/masterpage/Item_Cs_HoverPanel.js";
Notice that it points to a JS file, not the HTML file. And also that it’s located directly in the masterpage gallery and not in a subfolder (compared to the original Word html file).
The Item_Cs_HoverPanel needs no more changes than replacing Word with Cs and updating the title and description.
To add these Display Templates to the Design Manager, just drag and drop them into your browser where you downloaded the original Display Templates. The files will end up as Draft version so go ahead and publish both of the HTML files.
Once the files are approved, the Design Manager will create two JS files from these HTML files and that’s what’s used behind the scenes when rendering the search results.
Create a Result Type
The final thing we have to do is to tell the Search Center to actually use our newly crafted Display Templates when finding C# files. For that we need to create a new Result Type. Go to Site Settings and choose Search Result Types under Site Collection Administration. Click on New Result Type to start creating it. Give the Result Type an appropriate name, for instance “C# Files”. Under Conditions choose Show more conditions and select the FileExtension property and set it so that it “Equals any of…” and enter “cs” in the text box. Under Actions choose the C# item in the drop down and then simply click Save. You should now have a Result Type looking like this:
Test the stuff…
All that is left is now to test if all this works. Go to the search page of you Search Center and search for something you know exists in your C# files, that by now should have been indexed. You should then see something like this:
You see the items (using the Item_Cs Display Template) and if you hover over one of the search results then you see the preview from the WOPI Client being generated (using the Item_Cs_HoverPanel). Isn’t that nice, huh?
Summary
In this post you’ve seen yet another way to leverage a WOPI Client, this time in search, using a preview mode of the files. You have also seen how “easy” it is to generate a custom search result design using the Design Manager, Display Templates and Result Types. I’m really looking forward to seeing WOPI Clients for all kinds of different formats out there…