The SharePoint Enterprise Search Migration Tool (SMT), created by Microsoft, is a great little tool for moving/migrating search settings from one SharePoint Search Service Application to another, and even from a SharePoint 2007 SSP to a SharePoint 2010 SSA or FAST for SharePoint. The tool is available for download from the MSDN Archive - both as a binary and its source code. It is a console application that creates an XML when exporting the settings and uses the same XML when importing the settings, and it works great in a scripting environment. The SMT that's available from MSDN Archive allows you to migrate Best Bets, Search Scopes and Site Collection Search settings.
Introducing the Enhanced Search Migration Tool!
Consider using this tool when moving search settings from an SSA in a staging environment to an SSA in a production environment, or from production to a standby-farm. There's some stuff missing then - it's not enough just migrating best bets, search scopes etc. In my case I had to move the Managed Properties between our environments. Ok, you can script it all using PowerShell, but I'd prefer having something that automatically could move Managed Properties from dev to test to stage to production to standby.
So I took the source for the SMT and added functionality to work with the Managed Properties. It will export the Managed Properties from one SSA including its configuration and mapped crawled properties to the same XML file as the other Search Settings, and can then be used to import the Managed Properties to another SSA.
Here's how it works:
Export Managed Properties:
SearchMigrationTool.exe -export -managedproperty managedprops.xml
Import Managed Properties:
SearchMigrationTool.exe -import -managedproperty -conflictBehavior prompt managedprops.xml
As you can see it follows the same routine as the default SMT does and supports conflict resolution. And you can of course combine the export and import with the other SMT switches such as BestBet, Scope, SearchSettings or All (which exports/imports everything).
The features of this Enhanced SMT are:
- Import/Export Managed Properties, including all their configuration
- Only works for SharePoint 2010 to SharePoint 2010 (no 2007 and no FAST)
- Only maps existing crawled properties to the managed property
- It will not change the type of a Managed Property
- Handles the case when an SSA is present in multiple proxy groups (bug in original SMT)
Here's a sample on how to use it. Assume that your developers need to add a new Managed Property. Instead of them writing feature receivers or PowerShell to create the Managed Properties they just write a snippet of XML, like this:
The admins then use the Search Migration Tool to import this new Managed Property (as a part of the update POSH or similar).
Once this is done you will see the Managed Property in your SSA.
Can't be easier?
Support for this!
There is no support for this tool at all, but I would appreciate any feedback on it. I've had it running for a while amongst my friends and received positive feedback.
Hopefully my additions will be merged into the "official" SMT - but until then download the tool from here and get your SSA's in sync!
Here are the downloads:
Note! If you're going to modify/build the SMT you need to grab the SharePoint assemblies. They are not included in this package due to licensing restrictions.
I bet you will!
The Advanced Certification Team at Microsoft Learning will start a new Live Meeting series where you can learn more about the Microsoft Certified Master and Microsoft Certified Architect programs. It will be regularly held meetings where they will go into details about the programs. The program managers will make you understand the program mission and vision, how to prepare for a certification, how to apply for participation and the value of the programs. If you're interested in one or more of these programs I recommend you to attend one of the live meetings or watch the recordings. Of course attending the live meetings will allow you to directly ask questions to the program managers!
To register for one of the live meetings head on over to the Microsoft World Wide Events site and get your slot. Currently there are two planned sessions:
For a couple of weeks (ahem, months) I've been struggling with a strange Search Service Application issue. Some time back I went to check out on some Crawled Properties when making a tool to help copying settings between SSA's (more on this tool in another post). Then I noticed that there were tons of Crawled Properties with just garbled binary data(!) as the property name.
I searched like crazy for a while to find where these came from, there was nothing in the logs of any kind related to this. I could not locate any documents related to the Crawled Properties. I could not delete them, somehow they are connected to some content (at least that's what the system says), but there were no document samples. I created a new SSA and crawled the same corpus and the same (almost) corrupted junk crawled properties appeared.
A couple of days back I finally found out where they came from. With the grace from Microsoft support I did some queries on the SSA databases and found another crawled property that was inserted at the same time as these junk properties. This property had document samples! And those samples were e-mail .msg files. And specifically it was e-mails with encrypted content! I copied these files onto a brand new farm without any content and was able to reproduce the corrupted properties.
How to reproduce the corrupted Crawled Properties
To verify that this had to do with encrypted e-mail messages, and just not the ones I found; I encrypted an e-mail, sent it to myself and exported it as an .msg file. I took this .msg file and added it to a document library in a hot new VM (yea, I only got new ones since I had to rebuild all my VM's last weekend due to a corruption issue on one of my base images). Then I fired of a full crawl, with full logging enabled, and watched the Crawled Properties of the SSA. And as expected they showed up after just a minute.
So, be careful about having encrypted e-mail messages in your farm! Or prevent the issue...
How to prevent the issue
So, how do I get rid of these corrupted properties? Unfortunately there is no good (supported) way, at the time of this writing, except deleting your SSA and create a new one and before crawling the data remove the files or create a Crawl Rule.
If you already have these corrupted properties or if you want to prevent new corrupted properties you can create a Crawl Rule that excludes .msg files. That will help the situation - but you will not be able to search the .msg files (if they are encrypted you cannot search them anyways!).
These corrupted properties does not do any harm. You cannot use them and you don't notice anything else on the SSA except that they are there and annoys you. Only thing I can think of is that if you have a lots of them, you can run into trouble. There is a limit of 500.000 crawled properties per SSA! Sounds a lot, but for two .msg files I saw about 1.600 corrupted properties!
I hope this helps someone and I hope there will be a fix for this in the future - if that happens I'll update the post.
Visual Studio 11 Developer Preview is now available for download für alles and it does not only include the Windows 8 stuff like the previous preview did - this one contains the thing we all want - the SharePoint Developer tools.
Overall the performance of Visual Studio 11 is blazingly fast! I regret I tested it - since I will go back to 2010 tomorrow (or even tonight). They team has done a great job and included a lot of the PowerTools natively; such as the new Solution Explorer, the improved search feature etc.
But, back to the SharePoint Developer tools 11! What's new for all the kool kids! Here's a few highlights.
Content Type and List Designer
We all love our CAML, FieldRefs etc. But it isn't that productive. VS11 contains the long awaited Content Type Designer and also a very similar designer for Lists.
This is how the designer looks like when you're creating a Content Type:
When creating a list you have a great view to create Views:
Sandbox and Office 365 support
New features in the Office 365 space for Visual Studio was an easy bet. They have incorporated the things from the SharePoint Power Tools, such as better compile time support for the Sandbox etc. The best feature here is the Publish feature, which allows you to publish your package to a URL or local directory. You can now publish a SharePoint WSP directly to the sandbox in Office 365 - but you can't activate the solution from within Visual Studio.
But...you really need to test it out. I'm really looking forward to start working with projects in VS11!
It is always fun to get back on site after a couple of days off work. SharePoint 2010 is like an annoying little critter, if you're not there to cuddle with it it will do the most strange things.
I currently have a support case open regarding some issues with crawled properties (I hope that will be another story to tell another day) and went into the Search Service Application admin pages in Central Admin to check some things. When poking around I noticed that the incremental crawl hasn't been run for a few days - actually it stopped working on the 31st of December last year (sounds like ages ago now :-). In this farm we have three Search Service Applications and only this one hadn't been incrementally crawled - the other two worked just fine. I fired up an incremental crawl manually and that worked, waited for the next incremental crawl to start - and it didn't. Also tried a full crawl manually - which worked fine, but the scheduled crawls never started.
I searched through the trace logs and event logs and found nothing that I didn't expect to see.
I tried to Synchronize() the Search Service Instances - which I've previously done when similar stuff happened by running some PowerShell snippets on the servers in the farm. Nothing happened!
Ok, what is actually triggering the incremental crawls then? It is triggered by timer jobs called Indexing Schedule Manager on XXX, and there is one job per server. In this case we have two servers running search and I took a look at them and noticed that one of these jobs hadn't been run since the last scheduled crawl - while the other one had been running all the time (every 5 minutes).
I tried starting the timer job both through the Central Admin web UI and through PowerShell -but it just refused to start. Now this is strange! Now I took a look at the Timer Job History on this specific server and noticed that no timer jobs had been executed on that specific machine since that time when the scheduled crawl last was executed. Thankfully the trace logs were there and I could see that just after 2:00 AM on the 31st of December the Timer Service stopped working (no references to owstimer.exe at all since then). There's no warning or error or nothing that indicates that it would have stopped working (and the sptimerv4 service was still running on the machine). The only thing in there was a warning about a Forefront conflict. Nevertheless I needed to get it working and currently don't have time to spend on why this happened (if it happens again, then let's dig deeper).
I tried to do a restart-service sptimerv4 on the failing machine, without luck, it couldn't be gracefully shut down. So I actually had to kill the owstimer.exe process and then start the timer service.
Once the timer service was killed and started everything spun up and the incremental crawled kicked in after a couple of minutes and have been working perfect since then.
Interesting thing here was that I did not see any early warning signs, errors in the logs or similar, and the only way we found out that the Timer process had gone into a stale state was that the scheduled crawl for ONE of the Search Service Applications was not running.