Archives

Archives / 2011 / August
  • Yet another object to dispose correctly in SharePoint 2010 - SPUserSolution

    Tags: SharePoint 2010

    If you've been in the SharePoint business for a while (at least a couple of days) you should be aware of the SharePoint objects that needs to be properly disposed; SPSite and SPWeb in particular. Objects that need disposal inherits from the IDisposable interface and requires that you call the Dispose() method when you're done with the object  - this is to ensure that the object frees up resources that the .NET managed garbage collector cannot free up automatically. This includes objects such as non-managed SQL connections, resource handles, file handles etc. Disposing objects is nothing unique for SharePoint - all (real) .NET developers know how to dispose of a SQL connection. You can read more about the best practices around disposing SharePoint objects in the MSDN Disposing Objects article. Not doing this properly will eventually lead to application crashes, high memory usage and/or bad performance.

    The SPUserSolution object requires disposal

    If you are working with sandboxed or user solutions in code you've probably used the SPUserSolution object. This is another of these objects that needs your disposal-attention. A solution package (farm or user code) is essentially a cabinet file (compressed file) containing all the solution artifacts and assemblies. When activating a user code solution programmatically in the solution gallery you will get this SPUserSolution object which internally will contain a number of stream objects (also inheriting from IDisposable) and a non-managed file handle. These object must be disposed manually to avoid any resource or memory leaks. Here is a proper way to add and activate a user code solution

    byte[] bytes = ...;// grab the solution gallerySPDocumentLibrary solutions = (SPDocumentLibrary)newSite.GetCatalog(SPListTemplateType.SolutionCatalog);// add the solution to the gallerySPFile solutionFile = solutions.RootFolder.Files.Add("mysolution.wsp", bytes);// activate Solutionusing (SPUserSolution solution = newSite.Solutions.Add(solutionFile.Item.ID)) {    // do your thing...}

    Summary

    This was just a sample and a reminder that you need to be aware of what's happening when working with some of the SharePoint objects - make sure that you do your "Disposal dance" the correct way. The list goes long on what objects need to be disposed correctly - just check your objects if they inherit from IDisposable using Visual Studio or Reflector. Not all objects do any specific disposing though and some can be ignored - but better be safe than sorry.

    Tip: products such as CodeRush from DevExpress has some great built-in tools to directly identify objects inheriting from IDisposable and helps you make sure that they are properly disposed.

  • I'm on the SharePoint Pod Show talking about Web Parts

    Tags: Personal, Web Parts, SharePoint 2010

    The 65th SharePoint Pod Show is out featuring...tada...me :-)

    The SharePoint Pod ShowThe SharePoint Pod Show is THE podcast about SharePoint and is done by Rob Foster, Nick Swan and Brett Lonsdale and has featured a lot of great SharePointers from all around the world throughout the years. If you haven't already listened to the podcasts, then you got 65 episodes to catch up on! There are some epic ones, such as my favorite one #50 - which is about performance tuning. And make sure that you subscribe - you don't want to miss their SPC11 Road-trip...

    This episode was recorded at the MVP Summit earlier this year in Redmond between two sessions. It was my first time meeting most of the MVP's in real life and Rob was the one interviewing me. He's one heck of an interviewer asking questions like an ice-hockey radio commentator in some weird southern accent... It was great fun and we discussed SharePoint development and Web Parts development in particular. We talked through how to get started with SharePoint development, how to build Web Parts and why I wrote my book (SharePoint 2010 Web Parts in Action).

    Until next time.

  • Improve performance of your SharePoint 2010 applications using Windows Server AppFabric caching

    Tags: SharePoint 2010, AppFabric

    Besides SharePoint my very dear topics is performance optimizations and new technologies, so here's a post mixing all these together.

    Background

    Caching is one way to improve the performance of any application, there are several ways to do it in-memory, disk etc etc. SharePoint 2010 has a set of caching capabilities, most of them are in-memory caches and some involve disk or even SQL based. One problem with (especially) in-memory caching is that if you have a farm different servers may display different results, which is due to the fact that the different servers cached information at different times. Another problem with in-memory caching is that it's per process, that is that you have different caches for different web applications and application pools.

    Windows Server AppFabric is an extension to Windows Server. There's one cloud version of AppFabric in Windows Azure but there's also an on-premise installation of AppFabric that contains hosting and caching capabilities. AppFabric is a middle-tier service platform (the real middle-tier, not the other middle-tier :-) and can be used in all kinds of scenarios. The caching features of AppFabric is based on the Velocity project, which was one of my favorite acts for the PDC08 conference. It contains an easy to set up and configure distributed caching framework that can use SQL Server or disk as storage. In this post I will show you how to leverage the caching features of the on-premise version of AppFabric.

    Using a distributed cache such as AppFabric can solve many problems. I've already mentioned a few such as the problems with in-memory caching. Other interesting things is that you can easily set up cross farm caching or even cross domain caching. You can have the caching on separate servers in the application tier. You can share the cache between web applications and service applications. The cache isn't tied to the application pool so any recycles of that will not clear the cache and so on...

    Setting up and configuring AppFabric v1.1 CTP

    AppFabric v1.1 CTP is really easy to setup. You install the binaries (download 1.0 via Web Platform installer or 1.1 directly here) on the first machine to host the distributed cache and configure the cache. Then you add new servers to that cache cluster. Very much like you do when setting up a SharePoint farm. Follow these instructions on MSDN, you can't fail (which you can't say with SharePoint).

    Installing AppFabric

    Configuration is used either using the configuration application or using PowerShell (what else did you expect). In this case I configured it to use SQL Server as backend. Using the disk based would ideally be used in cross farm/domain scenarios (think about that - having data up to date cross farms, that's cool!). Once AppFabric is installed you need to start the AppFabric caching  using the following PowerShell commands. First use Use-CacheCluster to set the context and then just Start-CacheCluster to start the cache. There is one final configuration that is needed and that is to allow the application pool account to access the cache. This is done using the Grant-CacheAllowedClientAccount cmdlet. If you forget to set this you will see an ERRCA0017 error. AppFabric contains tons of configuration options. You could setup different clusters, named caches etc etc.

    Using the cache in a Web Part

    Now on to the SharePoint bits. In this scenario I'll build a Web Part that fetches quotes from Yahoo, that is doing an HTTP request to get a quote. This is a common scenario where you would build some kind of cache to avoid HTTP latencies. Another problem with HTTP requests is that they might time out or take a long time. You need to call it at least once so the poor bastard that's loading your page first might suffer from it. In this case we could actually create a timer job that makes the requests and put the data in the AppFabric cache before it's requested. But for this simple demo let's settle with a Web Part.

    Preparing your solution

    Before building the actual Web Part we need to set up the project to use the AppFabric cache. First two references must be added; Microsoft.ApplicationServer.Caching.Core and Microsoft.ApplicationServer.Caching.Client. They are found in the c:\Windows\System32\AppFabric\ folder (even though some documentation says otherwise). I could not directly add them as reference in Visual Studio, it could not find the directory, so I copied the file from that folder to a custom folder first.

    After adding the references we need to set up the configuration of our custom cache. Normally you do this using the application configuration file, but in SharePoint scenarios this doesn't work that well, so I'll do it all in code. In this sample I use hardcoded values for servers and ports - in a real world scenario you should of course make this configurable using for instance the P&P SharePoint Guidance Hierarchical Object Store. To access the cache we need to create a cache factory and retrieve our cache from that. Creating the cache factory object is an expensive operation so I'll implement this in a singleton thread-safe object to avoid creating the factory that many times. It looks as follows:

    sealed class CacheUtility {
        private static DataCacheFactory _factory;
        private static DataCache _cache;
        private static readonly object _lock = new object();
    
        CacheUtility() { }
    
        public static DataCache CurrentCache     {
            get {
                using (SPMonitoredScope scope = new SPMonitoredScope("DataCache.CurrentCache")) {
                    lock (_lock) {
                        if (_cache == null) {
                            List<DataCacheServerEndpoint> servers =                             new List<DataCacheServerEndpoint>();
                            servers.Add(new DataCacheServerEndpoint("SP2010", 22233));
                            DataCacheFactoryConfiguration config =                             new DataCacheFactoryConfiguration() {
                                Servers = servers
                            };
                            _factory = new DataCacheFactory(config);
                            _cache = _factory.GetDefaultCache();
                        }
                        return _cache;
                    }
                }
            }
        }
    }

    As you can see in this class I have the DataCacheFactory object and the actual cache object (DataCache) as static variables. They are only created on the first request to the CurrentCache property. The DataCacheFactory object is instantiated with a configuration, in this case I'll just add one AppFabric cache server using the server name and the cache port.

    Then let's add the Web Part to the project. In the Web Part class implement the function to get the stock quotes as follows:

    public class Quote {
        public string Name;
        public double LastTrade;
        public DateTime DateTime = DateTime.Now;
    }
    
    public Quote GetQuote(string ticker) {
        using (SPMonitoredScope scope = new SPMonitoredScope("GetQuote")) {
            using (WebClient client = new WebClient()) {
                string tmp = client.DownloadString(                String.Format("http://finance.yahoo.com/d/quotes.csv?s={0}&f=nl1", ticker));
                string[] arr = tmp.Split(new char[] { ',' });
                if (arr.Length == 2) {
                    return new Quote() {
                        Name = arr[0].Trim(new char[] { '"' }),
                        LastTrade = double.Parse(arr[1])
                    };
                }
            }
            return new Quote() {
                Name = "Not found"
            };
        }
    }

    It's a simple method, GetQuote(), that retrieves the stock quote using the Yahoo Finance API. If the quote is found it returns a Quote object with correct values and if not an "empty" Quote object.

    Also add a property to the Web Part so that we can turn on and off the caching:

    [Personalizable(PersonalizationScope.Shared)]
    [WebBrowsable]
    [WebDisplayName("Use Cache")]
    public bool UseCache { get; set; }

    Now it's time to implement the actual Web Part, and as always it's done in the CreateChildControls, like this:

    protected override void CreateChildControls() {
        using (SPMonitoredScope scope = new SPMonitoredScope("CachedWebPart.CreateChildControls")) {
            Quote quote = null;
            string ticker = "MSFT";
            if (UseCache) {
                DataCacheItem cacheItem = CacheUtility.CurrentCache.GetCacheItem("quote" + ticker);
                if (cacheItem != null) {
                    quote = (Quote)cacheItem.Value;
                }
            }
            if (quote == null) {
                quote = GetQuote(ticker);
                if (UseCache) {
                    CacheUtility.CurrentCache.Add("quote" + ticker, quote, new TimeSpan(0, 1, 0));
                }
            }
            this.Controls.Add(new LiteralControl(
                String.Format("Last trade for {0} is {1}, retrieved at {2}", 
                    quote.Name, quote.LastTrade, quote.DateTime)));
        }
    }

    This method is also pretty straight forward. It checks if the cache is enabled and if it is it tries to retrieve the quote from the cache using the GetCacheItem method. If it's not found it uses the GetQuote method to retrieve the quote and if the cache is enabled it stores the quote in the cache. Finally it just prints out the quote value. That's it!

    Test and evaluate

    So let's take a look at this Web Part now using the Developer Dashboard (notice the SPMonitoredScopes I added to the code). Here's how it looks without the cache enabled. The call to the Yahoo Finance API takes about half a second - a real performance killer.

    No caching

    Once we turn on the caching, the first call to the CacheUtility class will initialize the cache and for subsequent calls use that cached factory and cache. Notice how the first hit which creates the cache actually takes some time and in this case the HTTP request takes a really long time!

    Caching started

    Then once the cache is up and running, retrieving the quote takes almost no time.

    Caching in full effect

    Improve performance a little bit more...

    But it still takes about 10ms! This 10ms comes from that this is not an in-memory cache, it is actually stored in SQL Server. But hey, it's better than a couple of seconds. But I want to save an extra couple of cycles here. Ok, let's enable the client cache! Add the following row when configuring the cache factory and you will also have an in-memory cache that has a maximum number of object, a timeout and an invalidation policy.

    config.LocalCacheProperties = 
        new DataCacheLocalCacheProperties(
            100, 
            new TimeSpan(0, 3, 0), 
            DataCacheLocalCacheInvalidationPolicy.NotificationBased);

    In this case I set the invalidation policy to use notifications. To get this up and running you need to shut down the cache cluster and configure notifications for the cluster as follows:

    Stop-CacheCluster 
    Set-CacheConfig -CacheName "default" -NotificationsEnabled $true 
    Start-CacheCluster

    And now retry the Web Part and we'll see that this is more efficient:

    Client cache enabled

    Monitoring

    The AppFabric cache also contains a lot of features for monitoring and logging. The simplest command is the Get-CacheStatistics that show you stats of the cache:

    Cache stats

    Summary

    Next time you are thinking about caching in SharePoint (or any other .NET application for that matter) consider using AppFabric caching. It's a great and versatile middle-tier caching framework with near endless configuration possibilities. I know that this simple yet powerful framework can make a big difference in any of your projects. And why invent some own caching framework when we have something beautiful as this!

  • Stale Managed Metadata Databases in SharePoint 2010

    Tags: SharePoint 2010

    This is a short story about how you can get and resolve stale Managed Metadata Service (MMS) databases in SharePoint 2010. I've been working with Managed Metadata quite much and done some backup/restore juggling from production to test and to dev environments. Which by the way works really smooth. I've also recreated the MMS databases a couple of times.  After applying Service Pack and the re-released June 2011 CU I went into Central Administration to take a look at the databases and their upgrade status. This is what I found: four MMS databases, of which two didn't upgrade, and I only have two MMS service applications in this particular environment.

    Database status

    As you can see two of the databases reports "Database is in compatibility range and upgrade is recommended". There's no link to click to upgrade and there's no PowerShell command either to update them. These two databases were not connected to any MMS service application so it's no big deal, but who wants these kind of "warnings" in their CA.

    To create these stale MMS databases is an easy task; just go to Properties of the MMS SA and change the name of the database. This will provision a new and empty database for you, leaving the old database still registered in the configuration database of SharePoint. I don't know if this should be considered a bug or not, of course the database should not be deleted automatically, but should they remain registered as databases in the configuration database?

    So, in order to remove these databases, the easiest way is to fire up your favorite command line tool - PowerShell, and use a few lines of commands. This is how it's done:

    First of all the Get-SPDatabase command will return all registered in your farm.

    Get-SPDatabase

    By using the Id of the database it's easy to get one explicit database into a PowerShell variable like this:

    $db = Get-SPDatabase cb825c9d-8494-46fc-9ffc-3f81cffdfafc

    Now that we have one of our stale MMS databases we can use the Delete() method of the Database object to delete the actual reference in the configuration database, like this:

    $db.Delete()

    This will only delete the reference (the persisted object) and not the database. If you would like to remove the database as well, you can use the Unprovision() method prior to the deletion:

    $db.Unprovision()

    If you do just an Unprovision of the database, and don't delete the database, the database status in the upgrade status page will say "Not responding":

    Not Responding

    That was it, a quick and short post about stale Managed Metadata Service databases - how they become stale and how you remove them.

About Wictor...

Wictor Wilén is a Director and SharePoint Architect working at Connecta AB. Wictor has achieved the Microsoft Certified Architect (MCA) - SharePoint 2010, Microsoft Certified Solutions Master (MCSM) - SharePoint  and Microsoft Certified Master (MCM) - SharePoint 2010 certifications. He has also been awarded Microsoft Most Valuable Professional (MVP) for four consecutive years.

And a word from our sponsors...

SharePoint 2010 Web Parts in Action