SharePoint 2013: Personal Site instantiation queues and bad throughput

Tags: SharePoint 2013

In SharePoint 2013 the way Personal Sites (aka My Sites) are created have been totally rebuilt to support the new way of utilizing the Personal Sites. In this post I will go through how Personal Sites are provisioned, asynchronously, and bust a couple of myths about how interactive Personal Site instantiations should be “prioritized” and increase throughput.

Get the most out of SharePoint

Background

Personal Sites or My Sites were previously created “on-demand”. When a user went to his or hers non-existing My Site the provisioning started while the user waited for the site to be created, painfully watching the spinning animated gif. This worked fine in SharePoint 2010, and earlier, but now with SharePoint 2013 so much more are depending on the user having a Personal Site – everything from the social stuff in SharePoint 2013 (yes, not all SharePoint customers have yet wandered down the Yammer road) to the really great Office 2013 interaction (SkyDrive whatever). Requiring users to “manually” creating their Personal Sites is no longer a good option…

Note: this post is valid for SharePoint 2013 RTM – and hopefully some of the myths/bugs/features/stuff that is explained in this post will be fixed in upcoming releases…

Asynchronous Personal Site instantiation

In SharePoint 2013 the Personal Site is no longer created synchronously while the users wait for the site to be created. Instead all Personal Site provisioning are handled asynchronously by the Timer Job service (OWSTIMER.exe). So how does all this work? Whenever a user visits the My Site Host for the first time they are forced to create a Personal Site (depending on the My Site creation settings in the User Profile Service Application). The Personal Site instantiation are then added to one of possibly three queues (more about these in a bit). A set of timer jobs picks up the instantiation requests from these queues, creates the Personal Site and finally sends an e-mail to the user. The benefits of this new approach are many; the user can continue to do other stuff in SharePoint (real productive work instead of facetweeting for instance), the provisioning are not hogging the w3wp.exe process, we avoid performance bottlenecks, we should(!) see improved Personal Site creation throughput etc. There are of course drawbacks of this approach, such as it could potentially take some time before the personal site is created, since we have to wait for the timer job etc. But all in all this is something that is necessary (especially for the cloud) and something is very good.

The  Instantiation queues

I mentioned that there are three queues for the Personal Site instantiation.

  • The Interactive Request Queue
  • The Second Interactive Request Queue
  • The Non-interactive Request Queue

First of all – the difference between the Interactive and Non-interactive requests are basically that all “requests” to create a Personal Site through the web user interface are called interactive, while those requests coming from the Office client are non-interactive. The reason of having a second interactive request queue is that requests coming from the user interface – users actually wanting to facetweet – should have a higher priority and a higher personal site provisioning throughput.

SNAGHTML48ee1a

Each queue are (by default) polled every minute, by three timer jobs – one per queue, Each Timer Job will retrieve five (5) items from the queue and process them one at a time. According to some (ahem, SPC12) presentations this timer job are executed on every WFE, but that’s not what happens in real life. The Timer Jobs are based on the SPWorkItemJobDefinition Job Definition Type. This is a really nice timer job implementation that has a queue per content database. In this case the queue exists in the Content Database where the My Site Host is residing. This also means since the lock type of the job is Content Database per Web Application – only one server will run this timer job. So, to sum it up – for each queue and every minute a total of 5 Personal Sites can be created, which means about 300 Personal Sites per hour per queue – if you have the hardware that can handle that! One might think now that since there are two interactive queues, we can get a throughput of 600 Personal Sites an hour, well…no…

The second (and useless) interactive queue

The idea with the second interactive queue is that interactive personal site requests should have a higher throughput, and that is a good idea. But unfortunately something went wrong when this was implemented. Stop reading now if you don’t like digging deep into SharePoint internals and just accept the fact that this implementation is flawed…

The three queues, or rather the three Timer Jobs are created per Web Application through a hidden feature called MySitesInstantiationQueues. This feature creates the timer jobs and also configures an object in the config database (SPPersistedObject) which contains a value of how many queues that should be used. The funny thing is that this is an internal definition (the SPPersistedObject) and it sets the number of instantiation queues to 1, not 2. This means that we’re hardcoded to only have one instantiation queue, and the second interactive timer job just don’t do anything since it’s queue is always empty – even though we could use reflection to modify it, but then we’re in unsupported territory. So there goes the idea of bumping up the number of Personal Sites created asynchronously.

You can very easily see this in the ULS logs:

[Enque selection] GetWorkItemGuidHelper: Choosing first interactive queue instance as we are using only one queue.

Also, if we take a look directly at the configuration object we’ll see that it is configured only for one queue:

SNAGHTML7313d6

So the idea of having servers provisioning up to 600 Personal Sites an hour we instead have 300 Personal Sites per farm per hour. Imagine a new corporate portal launch spiced with social features for 20.000 users and about 5% of the users visiting the site at launch. This will give us 1.000 users in the queue and a potential queue time of 3:20 hours, at least – not that great in my opinion. Just imagine if you did some marketing of your new launch and only a couple of more users tried to use the social features…

Well, the good thing is that we don’t have the web servers hogged with creating thousands of Personal Sites, prohibiting real work.

Increasing the throughput anyways…

This section should really be classified under the unsupported things you shouldn’t do, ever, unless you need to… But for a large Enterprise Social deployment using SharePoint 2013 it might be a necessity, to not piss the end users off by having them waiting four hours to get started using SharePoint 2013.

If you don’t want to provision all the Personal Sites in advance, which in most cases are a dumb idea, you could instead of just relying on the queues to provision the Personal Sites you could read the queue yourself and create the personal sites “manually”. You need to find the database where the My Site is hosted and then query the ScheduledWorkItems table for items. Query for where the Type is E94A6CAA-B0F5-4897-B489-585CA50C7803 (which is the id of the first interactive instantiation queue, the second queue has it’s own Type Id – but you will never see that in here :-)). Find those with the InternalState equal to 8 – those are the ones waiting to be processed. Using this information you can use PowerShell or similar to spread out the load on other servers to create the personal sites.

SNAGHTML82acf1

To create the actual personal sites you could use the CreatePersonalSite() method of the UserProfile object, this method will bypass the queue and immediately create the site. 

Summary

By now you should be familiar with the process of how SharePoint provisions Personal Sites in SharePoint 2013. It’s a better approach than in previous versions. The idea of having multiple queues and allowing several servers to help out with provisioning the personal sites is good, but currently flawed. Hopefully this will get fixed in upcoming cumulative updates. As it is now it’s not working for a large enterprise rolling out SharePoint Enterprise Social, so you have to be very careful when planning a roll out of Enterprise Social in SharePoint 2013…or use Yammer…

10 Comments

  • Mahmoud Hamed said

    Hi Wictor,

    I think that if you create the the mysite manually this will note remove the workItem from the DB and the timerjob will use it. So what will happen if the timer job start to provision a site already provisioned by you manually ?!

    Regards,
    Mahmoud Hamed

  • Wictor said

    @Mahmoud.
    It will work just fine, when the queue actually picks it up it will just see that it is already created and continue on as normal. Of course it will only retrieve 5 items at a time, and if all of those already are created it will do nothing. I can NOT recommend removing it manually from the queue.
    I'm basing this reasoning on a real world scenario - where we had to utilize our app server tier to manually create those in the queue, since we had a couple of hours (6+ hrs) of queue time.
    /WW

  • Wictor said

    @Oliver - "just because you can doesn't mean you should". If you ask me officially I would not recommend that you remove them manually.
    /WW

  • Lake said

    What you will do is just creat a new User Profile Service,Don't use the default one ,try again.I just do it rencently.

  • Ulrik said

    Great article :)

    We would like to force setup of users MySite as soon as the user is provisioned in AD by our Control panel. Do you have any suggestions how to do that? We can script Powershell from our CPL App

    Best Ulrik

  • Johnathan Lightfoot said

    Thank you so much Wictor. The explanation you gave really helped me to understand personal sites in SharePoint 2013 and what is happening under the hood. Which led me to solving the issue I was experiencing!

  • Nelson Lamprecht said

    So, where is the page/control that adds items to queue? Basically, I want to understand how, once a user visits the mysite root page, sharepoint says, We're almost ready!, and then what is the connection between this page and the adding of items to the queue which you beautifully described?

  • Chandrasekaran said

    We are implementing social collaboration platform using SharePoint 2013 for a large enterprise client.

    We have decided to pre-provision the personal sites of all users using PowerShell. But, SharePoint throws the following error message when it trying to execute CreatePersonalSite method of the user profile

    "The site https://<<url>> could not be created. The following exception occurred: The field with Id {55b29417-1042-47f0-9dff-ce8156667f96} defined in feature {57311b7a-9afd-4ff0-866e-9393ad6647b1} was found in the current site collection or in a subsite"

    I found the below link having a workaround for this issue

    http://ricardocalejo.livethoughts.net/2013/04/24/workflow-service-store-and-workflow-task-features

    But, it seems to change the web template of SPSPERS, which is unsupported.

    Do you have any idea what going wrong with this approach?

    Thanks,
    Chandra

Add a Comment

AWS Tracker

About Wictor...

Wictor Wilén is a Director and SharePoint Architect working at Connecta AB. Wictor has achieved the Microsoft Certified Architect (MCA) - SharePoint 2010, Microsoft Certified Solutions Master (MCSM) - SharePoint  and Microsoft Certified Master (MCM) - SharePoint 2010 certifications. He has also been awarded Microsoft Most Valuable Professional (MVP) for four consecutive years.

And a word from our sponsors...

SharePoint 2010 Web Parts in Action