Archives

Archives / 2013 / March
  • SharePoint 2013 Managed Metadata field and CSOM issues in 2010-mode sites

    Tags: SharePoint 2013, SharePoint 2010, CSOM, JSOM

    Introduction

    SharePoint 2013 introduces a new model on how Site Collections are upgraded using the new Deferred Site Collection Upgrade. To keep it simple it means that SharePoint 2013 can run a Site Collection in either 2010 mode or 2013 mode, and SharePoint 2013 contains a lot of the SharePoint 2010 artifacts (JS files, Features, Site Definitions) to handle this. When you’re doing a content database attach to upgrade from 2010 to 2013, only the database schema is upgraded and not the actual sites (by default). The actual Site Collection upgrade is done by the Site Collection administrator when they feel that they are ready to do that and have verified the functionality of SharePoint 2013 (or you force them to upgrade anyways). But, the Site Collection admin might have to upgrade sooner than expected for some sites.

    Upgrade troubles

    This post is all about what has happened to us when our Office 365 SharePoint Online (SPO) tenant had been upgraded (and right now the Site Collection upgrade is disabled for us so we can’t do anything about it). The limitations of customizations in the 2010 version of SPO was very limited (to be kind) and we embraced JavaScript (which people have despised for a decade and now suddenly think is manna from heaven). We also leveraged Managed Metadata in a lot of lists, libraries and sites. We’ve built Web Parts using JavaScript CSOM to render information and also .NET CSOM stuff running in Windows Azure doing background work. Once our tenant was upgraded to 2013 all of the customizations using Managed Metadata stopped to work…

    JSOM sample with Managed Metadata

    I will show you one example of what works in SharePoint 2010 and what does not work in SharePoint 2013 when you’re site is running in 2010 compatibility mode.

    Assume we have a simple list with two columns; Title and Color, where Color is a Managed Metadata Field. To render this list using JSOM we could use code like this in a Web Part or Content Editor or whatever.

    var products;
    function loadProducts() {
    	document.getElementById('area').innerHTML = 'Loading...';
    	var context = new SP.ClientContext.get_current();
    	var web = context.get_web();
    	var list = web.get_lists().getByTitle('TestList');
    	products = list.getItems('');
    	context.load(products, 'Include (Title, Color)');
    	
    	context.executeQueryAsync(function() {
    		var collection = products.getEnumerator();
    		var html = '<table>'
    		while(collection.moveNext()) {
    			var product = collection.get_current();
    			html +='<tr><td>'
    			html += product.get_item('Title');
    			html += '</td><td>'
    			html += product.get_item('Color').split('|')[0];
    			html += '</td></tr>'			
    		}
    		html += '</table>'
    		document.getElementById('area').innerHTML = html;
    	}, function() {
    		document.getElementById('area').innerHTML = 'An error occurred';
    	});
    }
    ExecuteOrDelayUntilScriptLoaded(loadProducts, 'sp.js');

    As you can see a very simple approach. We’re reading all the items from the list and rendering them as a table, and this html table is finally inserted into a div (with id = area in the example above). This should look something like this when rendered:

    JSOM Rendering in SharePoint 2010

    The key here is that Managed Metadata in 2010 JSOM is returned as a string object (2010 .NET CSOM does that as well). This string object is a concatenation of the Term, the pipe character (|) and the term id. So in the code sample above I just split on the pipe character and take the first part. There was no other decent way to do this in SharePoint 2010 and I’ve seen a lot of similar approaches.

    Same code running in SharePoint 2013 on a SharePoint 2010 mode site

    If we now take this site collection and move it to SharePoint 2013 or recreate the solution on a 2010 mode site in SharePoint 2013. Then we run the same script, then this is what we’ll see, something goes wrong…

    Failed JSOM Rendering

    You might also see a JavaScript error, depending on your web browser configuration. Of course proper error handling could show something even more meaningful!

    JSOM Runtime exception

    Something is not working here anymore!

    What really happens is that the CSOM client service do not return a string object for Managed Metadata but instead a properly typed TaxonomyFieldValue. But that type (SP.Taxonomy.TaxonomyFieldValue) does not exist in the 2010 JSOM. Remember I said that SharePoint 2013 uses the old 2010 JavaScripts when running in 2010 compatibility mode. Unfortunately there is no workaround, unless we roll our own SP.Taxonomy.TaxonomyFieldValue class (but that’s for another JS whizkid to fix, just a quick tip to save you the trouble – you cannot just add the 2013 SP.Taxonomy,js to your solution).

    So why is this so then?

    If we take a closer look at what is transferred over the wire we can see that when running on SharePoint 2010 the managed metadata is transferred as strings:

    Fiddler trace on SharePoint 2010

    But on SharePoint 2013 it is typed as a TaxonomyFieldValue object:

    Fiddler trace on SharePoint 2013

    It’s a bit of a shame, since the server is actually aware of that we’re running the 2010 (14) mode client components! (SchemaVersion is what we sent from the CSOM and LibraryVersion is the library used on the server side)

    Fiddler trace on SharePoint 2013

    I do really hope that the SharePoint team think about this for future releases – respect the actual schema used/sent by the Client Object Model!

    Of course, this JavaScript solution will not work as-is when upgrading the Site Collection to 2013 mode. That is expected and that’s what the Evaluation sites are for.

    What about .NET CSOM?

    We have a similar issue in .NET CSOM, even though we don’t get a SharePoint CSOM runtime exception. Instead of returning a string object you will get back a Dictionary with Objects as values – but if you’re code is expecting a string you still get the exception. So in 99% of the cases it will fail here as well.

    Nothing exciting here, move along...

    Summary

    Deferred Site Collection update might be a good idea and you might think that your customizations will work pretty well even after an upgrade to SharePoint 2013, just as long as you don’t update your actual Site Collections to 2013. But you’ve just seen that this is not the case.

    Happy easter!

  • SharePoint 2013: Enabling PDF Previews with Office Web Apps 2013 March 2013 update

    Tags: SharePoint 2013, Office Web Apps

    In my last post (still smoking fresh) I showed you how to update your Office Web Apps 2013 farm to the March 2013 update, connect it to SharePoint 2013 and being able to view PDF documents in the browser. What I didn’t explain or show in that post was how to enable the PDF Previews in Search – but I’ll do it now.

    Pre-requisites

    Before you start fiddling with this, you need to make sure that you have the March 2013 update of Office Web Apps Server 2013 (WAC) installed and connected to your farm – if you don’t know for sure, ask your admins – sometimes they know…if they don’t give them the link to my previous blog post.

    Note: You don’t have to have a patched SharePoint 2013, this will work on the RTM bits.

    Default PDF Search Experience

    SharePoint 2013 natively supports crawling PDF documents, through the new document parsers. That is you don’t have to fiddle with any custom PDF IFilters etc. The native PDF document parser is a good enough solution, but have some room for improvements.

    The Search Experience and display in SharePoint 2013 is based upon Display Templates. Display Templates decide how the result should be shown and how the fly-out of the result should look like. For Office documents, when SharePoint 2013 is connected to a WAC farm, SharePoint displays inline previews which you can use to skim through the results really quick. For PDF this is not the case – not even if you use a WAC farm with the March 2013 update (even though the WOPI binding supports the interactivepreview action). This is a sample on how a PDF document could look like in SharePoint 2013 Search:

    Default PDF Search Experience

    Enabling PDF Previews in the Search Result

    Since we have the opportunity to modify the Display Templates and create our own Search Experience we can very easy modify the fly-out/hover panel of the PDF results to show the interactive preview. We can do this in two different ways…

    Create a new Result Type

    The easiest and fastest approach to enable our previews for PDF documents is by creating a custom Result Type. This is done by going to Site Settings > (Search) Result Types and then finding the PDF Result Type. Choose to Copy the Word Result Type. This will create a new Result Type.

    Copy the Result Type

    Give the new Result Type an appropriate name, “PDF with Preview” for instance. Then scroll down to Actions and in the “What should these results look like?” drop-down, choose to use the Word Display Template.

    Word or PDF?

    Then just click Save, go back to your Search Center and search for a PDF document and voilá – we have PDF Previews.

    PDF Previews in da house!

    Modifying the PDF Display Templates

    The second approach, which is a bit more advanced is to actually modify the PDF Flyout Design Template. You do this by going to Site Settings > Design Manager then choose 5. Edit Display Templates. Locate the PDF Flyout item by filtering on the Target Control Type column and use SearchHoverPanel, then scroll down to PDF Hover Panel. To modify the PDF Hover Panel, use the Word Hover Panel as a template. I’m not going in to all the details on how to modify the actual HTML file (see my similar post on how to achieve this here). But once you’ve modified it, make sure to save it to the gallery so the JS file is generated, and publish it. Now (if you made the correct updates) you will have Custom Search Previews for PDF documents.

    Summary

    You’ve now seen how you quick and easy can enable PDF Search Previews in SharePoint 2013 using the March 2013 update for Office Web Apps 2013. All you need to do is either create a custom Result Type and use the Word Display Templates or modify the default PDF Display Template. I really hope that Microsoft makes this a standard feature for upcoming SharePoint 2013 releases.

  • Office Web Apps 2013: Patching your WAC farm with no downtime

    Tags: Office Web Apps, SharePoint 2013

    I’m really glad to see some patches being rolled out for Office 2013, SharePoint 2013 and Office Web Apps 2013. There’s some really important fixes and some very interesting fixes that I’ve been waiting for. In this post we’ll take a look at the first Office Web Apps 2013 (WAC) update – specifically we’re looking at how to patch your WAC farm to minimize the downtime. If you follow my instructions you will have zero downtime (except for a brief moment where Excel stuff will not be accessible).

    Background and preparations

    RTM WACsFor this sample I will have a SharePoint 2013 (RTM) farm connected to a WAC 2013 (RTM) farm. The WAC farm is load balanced using NLB, like illustrated on the right.

    On order to update our WAC farm we need to download the March 2013 patch for Office Web Apps 2013. You can find it at Microsoft Download center and it is called “Update for Microsoft Office Web Apps Server 2013 (KB2760445)”. The KB article does not reveal much of what has changed, but if you take a look at an earlier patch KB2760486 (March 5th) you’ll notice some really cool things. If you don’t have time to read it – I’ll show you some of them at the end of this post. Ok, you do not have to install KB2760486, you only need the KB2760445 one.

    Patching the first WAC Server

    After downloading the patch and copying it to our WAC machines we can start the patching process. We would like to do this without the users even noticing it. First of all we need to take one of the servers out of rotation in the WAC farm. You do this using the load balancer. In my case I’m using Microsoft NLB and using PowerShell this is an easy task. Next thing to do is just remove the WAC server from the WAC farm – that is also a one-line PowerShell command. WAC Servers should only be patched with the binaries installed, not when they are participating in a WAC farm.

    These two lines does the trick:

    Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Stop-NlbClusterNode -Drain
    Remove-OfficeWebAppsMachine
    
    NOTE: You MUST remove the machines from the old farm before patching and then later create a new WAC farm (follow the instructions thoroughly). You cannot patch a running WAC Farm!

    Once this is done the server is no longer receiving requests and it’s no longer a part of the WAC farm and we can start patching this machine. All our end-user requests are now going to the remaining server(s) in the WAC farm.

    One RTM and one patching going on...

    To patch the machine start the downloaded .exe file (wacserver2013-kb2760445-fullfile-x64-glb.exe) and just follow the instructions; accept license agreement etc. You will be asked to close any PowerShell (IDE) windows that has the Office Web Apps module loaded and you might get asked to reboot once the update is finished (there’s of course no harm in doing it anyways).

    When the patching is done (and you rebooted) we need to create a new WAC farm, on the patched server. You do this using the same way as you did for the RTM version. Use the same server name as your old farm, same settings and the same certificate.

    Patching the rest…

    Once the new farm is created you have a few options depending on how many servers you have in your WAC farm. For instance if you have more than two, then you can take another one out of rotation, patch it and join it to the new and patched farm and then you flip the load balancer so it points to the new WAC farm. Depending on your load balancer you might have a brief moment where end-user request might be served by both versions. A bit funky situation but it works, as the illustration below shows:

    Dual versions...

    In my case I have two servers, so once I created the new farm I just put the patched server into the NLB rotation and then immediately take the RTM server out of rotation. Once the RTM WAC Server is taken out of rotation I remove WAC from that machine and start the patching process. All requests to the WAC farm are now on our patched server(s).

    One March 2013 and the other one is being patched

    As you can see, simple PowerShell operations this time as well, and this can (and should) be automated.

    # On the new patched WAC Server
    Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Start-NlbClusterNode
    
    # On the RTM WAC Server
    Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Stop-NlbClusterNode -Drain
    Remove-OfficeWebAppsMachine
    
    

    Once the other server(s) are patched, you need to join it to the newly created (and patched) WAC farm – just as you normally join a server to a WAC farm. And then you’re done. You’re farm is now patched and ready! But wait, there’s some stuff you have to do on the SharePoint side to enjoy all the new cool features.

    All new and shiny!

    Configuring new WOPI bindings

    One of the most interesting new features in the March 2013 update of Office Web Apps is the ability to view PDF documents! Yup, you heard me right! The March 2013 update contains a couple of new WOPI bindings that enable viewing and previewing (even mobile views) for PDF documents. These new WOPI binding has an application name called WordPdf. If you try to use the New-SPWOPIBinding cmdlet and just specify the WAC server name, you will get an error – since you already have those WOPI bindings. What you need to do is to specify the name of the application as an extra parameter to the command, like this:

    New-SPWOPIBinding - ServerName wac.corp.local -Application WordPdf

    WOPI Bindings

    Another approach could be to remove all the bindings and then add them all back again – of course this would cause a few seconds of downtime, and that’s not how I roll…

    Once you’ve run that command (and waited a couple of seconds) you can start previewing PDF documents in your Document Libraries. And now we got yet another good reason not to install the crapware called Adobe Reader!

    Viewing PDF documents

    Now, your end users can start taking advantage of all the new features in the March 2013 update.

    New Excel WOPI Bindings!

    If you’re examining the WOPI bindings provided by the March 2013 update really carefully, you’ll notice that the Excel application has six new bindings, three each for the two new actions syndicate and legacywebservice. We cannot use these two bindings in SharePoint 2013 RTM – the New-SPWOPIBinding cmdlet does not accept those actions. They are not a part of the well-known WOPI action values (as specified in the WOPI specification available when this post is written) – and I have at the moment no clue of what they are doing…stay tuned…

    Summary

    You’ve just seen how simple patching an Office Web Apps 2013 farm is. Compare that to patching SharePoint! By using the load balancer effectively you can make sure that the end-users can continue to use the Office Web Apps features while you’re doing the patching. Enjoy!

  • SharePoint 2013: Intelligent Promoted Results and Best Bets

    Tags: SharePoint 2013

    Introduction

    The search engine and search experience in SharePoint 2013 has been totally re-written, since its predecessors. In SharePoint 2010 we had something called Best Bets or Visual Best Bets if you worked with FAST for SharePoint. A best bet was a promoted result that was triggered by one or more keywords, used by the search admins to promote certain links or banners for specific search queries. In SharePoint 2013 this is now called a Promoted Results and the procedure of creating them is different and so much better – there’s more ways to trigger on, more ways to render the results etc, but the actual shown result isn’t that smart, until now…

    In this post I will show you how to create an even smarter and more intelligent Promoted Result – a best bet that actually uses the search query to do something interesting with. In this sample I will let users enter simple math questions and then we let the promoted result calculate the math question for you (just as the big search engines on the interwebz does).

    SNAGHTML5e5971

    Creating Promoted Results aka Best Bets

    A Promoted Result is created either on Search Service Application, Site Collection or Site level using the Search Query Rules option. To create a new Site Collection scoped Promoted Result you navigate to Site Settings > Search Query Rules. Next thing to do is to choose which scope to create this Query Rule for, choose Local SharePoint Results in this case. You have the option to further narrow down the Query Rule using segments and topic categories, but that’s not relevant for this exercise. Next thing to do is to click on New Query Rule. The Add Query Rule page contains quite a few options to make sweet things happening. First we have to give it a name and then we have to specify the query conditions. Query Conditions is what is used to trigger this promoted result. In order to trigger on our “mathematical” question we need to use a regular expression. To use a regular expression as the Query Condition you have to select Advanced Query Text Match and then write a regular expression.

    SNAGHTML6b29a7

    I’m no regular expression savant, but this regular expression works for this blog post. Anyone with a cooler or smarter or more complex regex – feel free to post it in the comments.

    \d+\s*[/\+\-\/*]\s*\d+

    Using a custom page as the Promoted Results

    Next up is to add the Promoted Results Action to this Query Rule. This is done by clicking on the Add Promoted Result further down on the Add Query Rule page. When adding a promoted result you will be asked for a Title, a URL and a description. In our case the URL is of interest – we want to render a page containing our logic to calculate the math query.

    I let the URL point to a page (an .aspx page we’ll create in a few seconds) and then I pass two query string parameters to it; IsDlg=1 to get rid of the chrome in the page and query={SearchTerms} to pass the actual query to the page. I also check the check box Render the URL as a banner instead of as a hyperlink so that the promoted result will be rendered as an iframe instead of a link.

    SNAGHTML70ca44

    Seem simple, doesn’t it, but wait – this will not work until you followed all the steps in this post. There is no thing out of the box that allows you to pass the search query to the Visual Best Bet.

    Now click Save on the Add Promoted Result page and then Save on the Add Query Rule page.

    Creating the custom promoted results page

    Now, before we test this (and it also takes a couple of seconds before this promoted result can be used – don’t ask me why!) let’s create our page that should calculate the result from the query. For this demo we create a fairly simple .aspx page with a JavaScript section and a small HTML snippet, as shown in the snippet below:

    <%@ Page Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage, Microsoft.SharePoint, 
    Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
    MasterPageFile="~masterurl/default.master" Language="C#" %> <asp:Content ContentPlaceHolderID="PlaceHolderAdditionalPageHead" runat="server"> <script type="text/javascript"> function getQueryStringValue(name) { var qs = window.location.search.substring(1); var pairs = qs.split("&"); for (var i = 0; i < pairs.length; i++) { var pair = pairs[i].split("="); if (pair[0] == name) { return pair[1]; } } return ''; } _spBodyOnLoadFunctions.push(function () { var query = getQueryStringValue("query") try { document.getElementById("result").innerHTML = eval(query) } catch (e) { document.getElementById("result").innerHTML = "error" } }); </script> <style> #s4-ribbonrow { display:none; } </style> </asp:Content> <asp:Content ContentPlaceHolderID="PlaceHolderPageTitleInTitleArea" runat="server"> Intelligent Best Bet </asp:Content> <asp:Content ContentPlaceHolderID="PlaceHolderMain" runat="server"> <h1>SharePoint Search Calculator</h1> <h2>The result is: <span id="result"></span></h2> </asp:Content>

    The first section contains a simple JavaScript that extracts the Search Query from the query string and then uses the JavaScript eval() method on it to “calculate” the value. In the PlaceHolderMain I added a span element which is where the calculator writes the result.

    So let’s upload this page to the Documents library in the Search Center – which was the location I specified when creating the Promoted Results. If we now search for something that should be triggered by the Query Rule you will see the page rendered as a banner, but  it will not calculate the query as we expected.

    Passing the search query to the custom promoted results page

    So, why aren’t our result rendering properly? It all comes down to that the default Display Template for Best Bets/Promoted Results does not pass the search query to the iframe. In order to fix this we need to modify this Display Template. The Display Template that we need to edit is the Item_BestBet.html file.

    First we add two lines, directly following the var isVBB = … line, like this

    var isVBB = ctx.CurrentItem.IsVisualBestBet;
    
    var url = ctx.CurrentItem.Url
    url = url.replace('{SearchTerms}',ctx.CurrentItem.ParentTableReference.Properties.QueryModification)
    

    The code is crafting the URL to use in the iframe by retrieving the URL from the CurrentItem and then replacing the text {SearchTerms} with the actual search terms used.

    Next we need to modify the line where the iframe is rendered.

    <iframe id="_#= $htmlEncode(id + Srch.U.Ids.visualBestBet) =#_" 
        class="ms-srch-item-visualBestBet" title="_#= $htmlEncode(ctx.CurrentItem.Title) =#_" 
        scrolling="no" frameborder="0px" 
        src="_#= $urlHtmlEncode(url) =#_"></iframe>
    
    Modify the src attribute of the iframe element so that it uses our new url property instead of the default URL value from the current context. Save the file and go back to the Search Center…

    See it in action

    Now we can take it for a test drive. Search for simple mathematical questions and see the results being rendered as a promoted result.

    SNAGHTML80e9e4

    Summary

    I’ve just shown you how to make the Promoted Results aka Visual Best Bets more intelligent by creating a Query Rule that promotes a web page that does operations on the search query. To get this all to work the secret ingredient was to modify the Display Template for promoted results to allow it to pass the search query to the promoted results page.

    I can really see this being extended to provide the users with even more interesting “intelligent” promoted results.

  • SharePoint 2013 Central Administration Productivity Tip

    Tags: SharePoint 2013

    Here’s a short post with a tip that I find very useful.

    In many scenarios you have several SharePoint 2013 installations to handle – it might be different farms, production environments, testing, staging, development etc. Do you know which Central Administration you’re working in at the moment? They all look the same, SharePoint Blue, the regular Status Bar warning that you’re running out of disk space etc. Unless countermeasures are taken you don’t know what environment you’re in unless you take a look at the URL – which in many cases is just another server name and port. It’s very easy to make a mistake and make a change in the production environment instead of in the test or dev environments.

    So, how do we keep track of what Central Administration site we’re actually working on at the moment? One way could be to change the theme of the Central Admin site. But, was the production CA red or was it the one with the dog in the background? I’ve got a better tip for you!

    Here’s how I’ve done to keep track of the Central Administration sites. I take advantage of the Suite Link Bar in the upper left corner. By default it says just “SharePoint” – yes we know it’s SharePoint.

    Standard Suite Bar

    By modifying the Web Application property that controls this text we can easily change it to something more friendly and appropriate for the specific farm, like below.

    Cool Suite Bar

    It’s a very simple PowerShell operation to accomplish this. You just retrieve the Central Administration Web Application object, then update the SuiteBarBrandingElementHtml property and set it’s value to something that tells you which Central Administration site this is:

    asnp microsoft.sharepoint.powershell
    $ca = Get-SPWebApplication -IncludeCentralAdministration | `
        ?{$_.IsAdministrationWebApplication -eq $true}
    $ca.SuiteBarBrandingElementHtml = "<div class='ms-core-brandingText'>Central Admin: FarmA Production</div>"
    $ca.Update()

    You should leave the div element with the ms-core-brandingText class, to get decent formatting of the text, but inside it you can add whatever HTML you like (I’m thinking the marquee or blink tag…)..

    That’s it. I hope I saved a few kittens from being slaughtered…

  • SharePoint 2013: Personal Site instantiation queues and bad throughput

    Tags: SharePoint 2013

    In SharePoint 2013 the way Personal Sites (aka My Sites) are created have been totally rebuilt to support the new way of utilizing the Personal Sites. In this post I will go through how Personal Sites are provisioned, asynchronously, and bust a couple of myths about how interactive Personal Site instantiations should be “prioritized” and increase throughput.

    Get the most out of SharePoint

    Background

    Personal Sites or My Sites were previously created “on-demand”. When a user went to his or hers non-existing My Site the provisioning started while the user waited for the site to be created, painfully watching the spinning animated gif. This worked fine in SharePoint 2010, and earlier, but now with SharePoint 2013 so much more are depending on the user having a Personal Site – everything from the social stuff in SharePoint 2013 (yes, not all SharePoint customers have yet wandered down the Yammer road) to the really great Office 2013 interaction (SkyDrive whatever). Requiring users to “manually” creating their Personal Sites is no longer a good option…

    Note: this post is valid for SharePoint 2013 RTM – and hopefully some of the myths/bugs/features/stuff that is explained in this post will be fixed in upcoming releases…

    Asynchronous Personal Site instantiation

    In SharePoint 2013 the Personal Site is no longer created synchronously while the users wait for the site to be created. Instead all Personal Site provisioning are handled asynchronously by the Timer Job service (OWSTIMER.exe). So how does all this work? Whenever a user visits the My Site Host for the first time they are forced to create a Personal Site (depending on the My Site creation settings in the User Profile Service Application). The Personal Site instantiation are then added to one of possibly three queues (more about these in a bit). A set of timer jobs picks up the instantiation requests from these queues, creates the Personal Site and finally sends an e-mail to the user. The benefits of this new approach are many; the user can continue to do other stuff in SharePoint (real productive work instead of facetweeting for instance), the provisioning are not hogging the w3wp.exe process, we avoid performance bottlenecks, we should(!) see improved Personal Site creation throughput etc. There are of course drawbacks of this approach, such as it could potentially take some time before the personal site is created, since we have to wait for the timer job etc. But all in all this is something that is necessary (especially for the cloud) and something is very good.

    The  Instantiation queues

    I mentioned that there are three queues for the Personal Site instantiation.

    • The Interactive Request Queue
    • The Second Interactive Request Queue
    • The Non-interactive Request Queue

    First of all – the difference between the Interactive and Non-interactive requests are basically that all “requests” to create a Personal Site through the web user interface are called interactive, while those requests coming from the Office client are non-interactive. The reason of having a second interactive request queue is that requests coming from the user interface – users actually wanting to facetweet – should have a higher priority and a higher personal site provisioning throughput.

    SNAGHTML48ee1a

    Each queue are (by default) polled every minute, by three timer jobs – one per queue, Each Timer Job will retrieve five (5) items from the queue and process them one at a time. According to some (ahem, SPC12) presentations this timer job are executed on every WFE, but that’s not what happens in real life. The Timer Jobs are based on the SPWorkItemJobDefinition Job Definition Type. This is a really nice timer job implementation that has a queue per content database. In this case the queue exists in the Content Database where the My Site Host is residing. This also means since the lock type of the job is Content Database per Web Application – only one server will run this timer job. So, to sum it up – for each queue and every minute a total of 5 Personal Sites can be created, which means about 300 Personal Sites per hour per queue – if you have the hardware that can handle that! One might think now that since there are two interactive queues, we can get a throughput of 600 Personal Sites an hour, well…no…

    The second (and useless) interactive queue

    The idea with the second interactive queue is that interactive personal site requests should have a higher throughput, and that is a good idea. But unfortunately something went wrong when this was implemented. Stop reading now if you don’t like digging deep into SharePoint internals and just accept the fact that this implementation is flawed…

    The three queues, or rather the three Timer Jobs are created per Web Application through a hidden feature called MySitesInstantiationQueues. This feature creates the timer jobs and also configures an object in the config database (SPPersistedObject) which contains a value of how many queues that should be used. The funny thing is that this is an internal definition (the SPPersistedObject) and it sets the number of instantiation queues to 1, not 2. This means that we’re hardcoded to only have one instantiation queue, and the second interactive timer job just don’t do anything since it’s queue is always empty – even though we could use reflection to modify it, but then we’re in unsupported territory. So there goes the idea of bumping up the number of Personal Sites created asynchronously.

    You can very easily see this in the ULS logs:

    [Enque selection] GetWorkItemGuidHelper: Choosing first interactive queue instance as we are using only one queue.

    Also, if we take a look directly at the configuration object we’ll see that it is configured only for one queue:

    SNAGHTML7313d6

    So the idea of having servers provisioning up to 600 Personal Sites an hour we instead have 300 Personal Sites per farm per hour. Imagine a new corporate portal launch spiced with social features for 20.000 users and about 5% of the users visiting the site at launch. This will give us 1.000 users in the queue and a potential queue time of 3:20 hours, at least – not that great in my opinion. Just imagine if you did some marketing of your new launch and only a couple of more users tried to use the social features…

    Well, the good thing is that we don’t have the web servers hogged with creating thousands of Personal Sites, prohibiting real work.

    Increasing the throughput anyways…

    This section should really be classified under the unsupported things you shouldn’t do, ever, unless you need to… But for a large Enterprise Social deployment using SharePoint 2013 it might be a necessity, to not piss the end users off by having them waiting four hours to get started using SharePoint 2013.

    If you don’t want to provision all the Personal Sites in advance, which in most cases are a dumb idea, you could instead of just relying on the queues to provision the Personal Sites you could read the queue yourself and create the personal sites “manually”. You need to find the database where the My Site is hosted and then query the ScheduledWorkItems table for items. Query for where the Type is E94A6CAA-B0F5-4897-B489-585CA50C7803 (which is the id of the first interactive instantiation queue, the second queue has it’s own Type Id – but you will never see that in here :-)). Find those with the InternalState equal to 8 – those are the ones waiting to be processed. Using this information you can use PowerShell or similar to spread out the load on other servers to create the personal sites.

    SNAGHTML82acf1

    To create the actual personal sites you could use the CreatePersonalSite() method of the UserProfile object, this method will bypass the queue and immediately create the site. 

    Summary

    By now you should be familiar with the process of how SharePoint provisions Personal Sites in SharePoint 2013. It’s a better approach than in previous versions. The idea of having multiple queues and allowing several servers to help out with provisioning the personal sites is good, but currently flawed. Hopefully this will get fixed in upcoming cumulative updates. As it is now it’s not working for a large enterprise rolling out SharePoint Enterprise Social, so you have to be very careful when planning a roll out of Enterprise Social in SharePoint 2013…or use Yammer…

About Wictor...

Wictor Wilén is a Director and SharePoint Architect working at Connecta AB. Wictor has achieved the Microsoft Certified Architect (MCA) - SharePoint 2010, Microsoft Certified Solutions Master (MCSM) - SharePoint  and Microsoft Certified Master (MCM) - SharePoint 2010 certifications. He has also been awarded Microsoft Most Valuable Professional (MVP) for four consecutive years.

And a word from our sponsors...

SharePoint 2010 Web Parts in Action