2/3/2010In a discussion on twitter yesterday a number of people expressed their opinions on how best to run SharePoint in a dev environment or simply on your laptop. People are looking for easy to run and configure and minimize memory with validation, development, demos, and prototyping as goals. Quick Links to the SharePoint VHDs and Bits Understanding the Debate A lot of frustration was expressed at Microsoft’s virtualization desktop strategy in how it appears to be lacking. Most of the frustration is specifically pointed at both Virtual PC and Virtual Server not supporting x64 or any 64bit guest OS for that matter. Microsoft’s current released x64 virtualization strategy has put all the eggs in the Hyper V basket without support for desktop OS’s like Windows 7. We don’t like running Windows Server on our laptops because there is NO hibernate or sleep. Very inconvenient for meetings and power consumption if you simply close the lid. Windows Server 2008 really wasn’t designed for laptops. Eli Robillard, SharePoint MVP in Toronto explains this great debate in his post How to build a SharePoint 2010 dev environment, “Windows 2008 R2 with Hyper-V is a good choice for virtualization on desktop or server installs, but is not supported by Microsoft for use on laptops. Hyper-V was really never intended for use on laptops, and so conveniences like Sleep and Hibernation aren’t there. Other notable things not usable on Windows 2008 R2 are Bluetooth, Zune software and Call of Duty, and there are others. I really like Windows 7 but I don't want to do development on my daily-use OS, so third-party virtualization like VirtualBox or VMWare are the clear winners for me, both of which support a 64-bit client OS. That said, many of the SharePoint MVPs who presented at SharePoint Conference 2009 (#spc09) use a configuration that can dual-boot into either Windows Server for development and presentations, or into Windows 7 for personal use.” There are so many options now, and it’s confusing to know which to use. Is there a right answer? Not sure yet, let’s look at this. You got your new laptop or new desktop hardware and want to way out the options. Make sure you’re aware of common issues: 10 Tips Troubleshooting Installations for SharePoint 2010 and SharePoint Foundation. Becky has a good list from her experience as well SharePoint 2010 Knock Out edition What You Need to Know!. Including known issues with beta 2 SharePoint 2010 from the SharePoint team blog and for Known Issues Download List for FAST. I wanted to simply get this out there and let you know what my experience has been. Experience is definitely all over the map based on the conversation on twitter. To understand SharePoint 2010 Virtualization in a production environment you should view the recent presentation on SharePoint 2010 Virtualization Farm Architecture by Michael Noel, MVP. The Laptop Options for Running SharePoint 2010 Assumptions: 1. SharePoint 2010 2) 64 bit 3) Need for portability/laptop | Platform | Hibernate | NON MS | License | Virtual | Physical | | Win7 with SP | x | | | | x | | W2K8 with SP | | | | | x | | Dual Boot Win7 and W2K8 | x / | | | | x | | Win7 Boot to SP VHD | | | | * | | | Win2K8 R2 Boot to SP VHD | | | | * | | | W2K8 with HyperV SP VHD | | | | x | | | Win7 with VMWare SP | x | x | x $ | x | | | Win7 with VirtualBox SP | x | x | | x | | - Win 7 with SP bare metal – Code Project: Running SharePoint 2010 on Windows 7 x64 ; Upside: Very accessible and quick to get in and out of. Downside it’s always running in the background eating up RAM. Other desktop operations will be slower as a result, UNLESS you shut down the services <updated 2/4/10> which is explained in this post by Emmanuel: Stop and Go with SharePoint on your workstation. (Note the .zip attachment in the post) Thanks for the tip @harbars. As well check out Spence’s Tips and Tricks for running SharePoint 2010 on Win 7.
- W2K8 with SP bare metal - MOSS How to: Installing SharePoint 2010 Beta on Windows 2008 : Upside You’re running server! Great for development environment Downside: Not good for a desktop environment. Many desktop apps aren’t written for server, while they may or may not work. Some drivers have issues as well. It’s a you figure it out what works and doesn’t. Better have a server license or have a TechNet/MSDN subscription.
- Dual Boot Win 7 and W2K8 – Upside: Your desktop environment is clean. Reboot to easily get into your SharePoint or Desktop. Downside: You have to reboot every time you want to get into your SharePoint environment. Not a good option for a developer. Ok for demos.
- Win 7 Boot to SP on W2K8 R2 VHD - Sharepoint 2010 Development environment – Windows 7 boot VHD ; Upside Dual boot and easy to rollback and reconfigure if you want to update the image. It’s a VHD. Very cool. Downside: You have to reboot every time you want to get into your image and go back to do your desktop apps. Ok for demos, but rough on a developer. (* Update: In the comments Todd argues that boot to VHD isn’t virtualization. I understand his point, even though the V in VHD stands for virtual)
- Win2K8 R2 Boot to SP VHD - Pete Rodriguez's Blog : Running SharePoint on a native boot VHD – Upside: Dual boot and easy to rollback and reconfigure if you want to update the image. It’s a VHD. Very cool. Downside: You have to reboot every time you want to get into your image and go back to do your desktop apps. Ok for demos, but rough on a developer. (* See point for for more details.)
- W2K8 with HyperV SP VHD – CJG: SharePoint 2010 - Step by Step Install (HyperV with Full SQL) Upside: Most likely to mimic what you’ll do in production and image should be compatible to copy from test environment. You can shut down the image when you’re not using it. As well if you were to do this with Windows Server 2008 Core, it makes sense from a security perspective and sounds very clean. Easily swap out the images. Downside: You’re always running 2 OS’s to do anything. Takes a lot of RAM. Both Host and Guest must be 64 bit. With W2K8 as your desktop there are definitely downsides to what won’t work including lack of hibernation. Hyper V is a lot of management for a simple image.
- Win 7 with VMware Workstation or ESX or Fusion, etc.. with SP - Faizal: Step by Step SharePoint 2010 Installation Guide (Standalone on VMware Workstation) Upside: Great management UI that’s easy to use and has support behind the solution. Well understood by the industry. Supports 32 or 64 bit host and 32 and 64 bit host. Current virtualization Leader. Downside: It’s not bare metal so you’re running both your client and the server requiring more memory. Microsoft would prefer you not talk about this one as the best one, but based on my experience, it is extremely common. Cost - It’s the paid solution and a competitive one at that. VMWare Workstation is $189 (go to VMWare for the real details or to get a 30 day trial), but many IT departments already have licensing for it.
- Win 7 with VirtualBox SP – Eric Harlan: Setting up your Virtual Machine for SharePoint 2010 (VirtualBox) Upside: Free personal use virtualization, turn it on when you need it, shut it down when you don’t. I think the major battles between Sun and MS are pretty much over as any recent news is about virtualization and partnering, so I don’t think this offends too many people. It is also very slim and very simple to setup and configure. It also works with Hyper V images, a huge bonus. Supports 32 or 64 bit host and 32 and 64 bit host. (Quick Hint: Make sure you choose Windows Server 2003 x64 if you create or add through the interface.) It’s an easy download from Virtualbox.org. Very handy and feels light weight. Downside: It’s not fully an MS Solution, it’s not your desktop, and it’s Sun. Does politics stop you? I saw an image for oracle on the homepage of virtualbox.org oooooh Sometimes.
Common reference on TechNet: Setting Up the Development Environment for SharePoint Server What I’ve done… I’ve setup a 64bit Win7 machine and failed silently I have to assume due to the 2GB RAM that was on it. I’ve successfully used HyperV to manage a number of 2010 images including some I’ve built on a tower at home. :) I’ve briefly used VMware (during the Tech Preview of 7). I currently am using VirtualBox and will be using it next week at SPTechCon. I’ve setup that laptop with boot to VHD, but need to first upgrade that image with R2, so it will work. Loving my new Dell Latitude 14in laptop with backlit screen 8GB RAM with all the bells and whistles (minus the solid state drive I need or want… need!). Thanks Todd and Eric for that discussion in Sweden. I’ll be sure to blog about the boot to VHD experience, I’ve already captured a lot of lessons and screenshots. It could end up being the most simple solution in the long run, but not a long term solution either. Conclusion. The debate continues. This poll that speaks 1000 words… What virtualization technology are you using for evaluating SharePoint 2010 Beta2? 
The count which should be added is 4 for Boot to VHD and I think that would be higher if formalized as an answer. This is a snapshot and expect this to get more votes by Mar 31 when the poll ends. Commentary: In the running neck in Neck is VMWare in the lead (especially if you combine VMWare solns) and MS HyperV with VirtualBox not far behind… Be sure to add your vote to the poll! (Thanks @Chakkaradeep) Let me know if you think this debate is much simpler than I think! Feel free to add your thoughts and comments and help us get to a happy place. I think we’d all be happy to see Microsoft Virtualization team come up with a laptop/desktop virtualization strategy for Windows 7 to support x64 especially if it could support 64 bit guest OS on a 32 or 64 bit host. (I know they are listening! :)) 2/2/2010I’ve been thinking a lot about the SQL 2008 R2 and SharePoint 2010 better together story and wanted to capture some of what’s been floating around in my head. I haven’t seen enough content and blogs on this stuff and hope that this will help spur more conversations on the topic. Mike Watson has put out a couple of well put together posts on SQL 2008 and SharePoint 2010 in his storage considerations post and on SharePoint 2010 high availability improvements. I also put some thoughts on why SQL 2008 for SharePoint from a while back. With the product team’s announcement of requirements of 64 bit SQL and SQL 2005 and SQL 2008, many are asking questions. Having spent time with SQL 2008, I do see it as a no brainer to skip 2005 or to simply go with the gold and do the right thing from the beginning. A lot of the high availability (mirroring) and security features (transparent data encryption) are in the enterprise version, so be careful to do the right thing. Ironically I did run into 10 Things DBAs will love about SharePoint 2010 which only enriches this story which focuses on the SQL side even more. I have been using SQL 2008 R2 on all of my SharePoint 2010 installs. Love the new features “Top 10 Features of SQL 2008 R2” 1. Mirroring Aware – Yeah, right in the SharePoint 2010 box you get the ability for SharePoint to understand primary and secondary content databases. Configure it right in Central Admin. Woohoo! Downside, SharePoint doesn’t setup the mirrored pair, and it doesn’t set this up automatically for new databases. It requires you or your SQL buddies to setup the mirroring and then tell SharePoint about it. Definitely a step in the right direction. The fact that the Indexing support is redundant now makes the full story truly highly available. That difficulty around what to do with the SSP for disasters has been cleaned up a lot making the story better for SQL and SharePoint people alike. 2. SharePoint Patching greatly increases HA for SQL – You wouldn’t think patching would have any dependencies on SQL, but the great news here is the ability to upgrade the binaries across your SharePoint Servers and not make the entire farm unavailable. Your binaries can simply be out of sync with your databases for a limited time, essentially you can patch for security, and then update your databases during your scheduled maintenance window. This allows you to keep your SLAs and prevent patching from causing unnecessary downtime a big win with SharePoint 2010. Mike talks about the patching in his HA for 2010 post. There was even a FULL session at SPC09 (SharePoint Conference) on patching! Also in his post pay attention to what’s happened with the Index service due to the enhancements with the FAST search integration and architectural changes. HUGE. 3. SQL content database structure – The structure in the content database has been redesigned. The underlying way it stores the blobs has been enhanced. I’m sure more info will come out, but this was a major step in the way of scale and reducing blocking. By simply upgrading to SQL 2008 a lot of the blocking went a way due to the nature of the way SQL 2005 did it’s row locking. 64bit also helps, and since that is required we see better scalability for both numbers of databases and size. Instead of having all the blobs in one single filestream table it is now better distributed which significantly increases the scalability of the databases as well as provides the ability to split the structure. Don’t think this means a site collection can be split across databases. Not yet. SharePoint Sandbox goes into more of the structural db changes. 4. SQL content database file group support – With the structural changes in the way content is stored in the database. (That’s right your data is still in the database by default.) You can now setup file groups on your content database in extreme cases. The average environment won’t care, but they will still take advantage of the structural improvements. Those in the really extremely large case, such as the more than 200-300GB you will now be able to split your content database across LUNs, or volumes to both increase performance and to simply grow the environment as it needs to. (Still not a best practice, but with the extremely large, you’ll find TB repositories with the new scaling features.) 5. More Specialized databases – Where did all these databases come from? Before you get upset, it’s actually a good thing, not only for Microsoft, but for you. With the tens of new databases that we have we can now focus the write intensive databases on their own LUNs and put the read intensive databases on theirs. Why not have Microsoft do the right thing by default? They don’t have a clue whether your environment is a collab, social, portal, or BI solution. You will know best if your app will focus on search, or as an application, and by having databases be specialized through the new service apps you will be able to do the right thing with the right databases. Don’t provision what you don’t need, and allow your applications to become specialized and do the right thing from a disk perspective. The profile database even has options for synchronization a much needed enhancement. It isn’t managed on the SQL side, so don’t worry about that. There are architectural decisions about where to put what service and consuming those services for some services is even possible across the WAN! That’s pretty huge. 6. Remote blob storage enhancements – While the scenario around migrating your WIDE (Windows Internal Database Engine) to the SQL Express with 4 GB limit sounds kind of wierd and scary in SharePoint 2010, right out of the box with in place upgrade Microsoft tries to do the right thing and move your blobs into remote storage with the configuration and lists in the database and the data outside the database. The enhancements around RBS and EBS in SharePoint 2010 still require an RBS provider. There is support for transparent Remote blob storage. This blog post on externalizing SharePoint content in SharePoint 2010 captures the meat of it in a recap of the SharePoint Conference session by MS people from the SQL advisory team. It has some great detail around how it has been enhanced and some great considerations. Enhancements in SQL 2008 R2 handles blobs way better, and enhancements in SharePoint 2010 make using remote blob storage with vendors a more manageable proposition as well. I highly caution people against using RBS to simply save on management costs. If you’re going to do it, do it for disk storage costs. Why? Because it isn’t easier to manage outside of SQL, it’s harder and more complex. Really! There is a free RBS filestream provider from MS, but I refer to above with use with SharePoint 2010 (Not in the box), but don’t go down that route unless you really know what you’re doing. Again I refer to the post above. Not many have yet gone this route without support from a vendor. Caution. 7. Throttling – List and performance throttling reduces blocking in the database. While these features are exposed to the SharePoint administrator… conversations around what blocking should look like and where to set the new thresholds is a good conversation to have. The throttling on lists should definitely mean we’ve found a solution to the blocking list. Large site deletes are another matter entirely, but on the surface, the common scenarios are now much more manageable. I’ve put together a number of blogs on the throttling. 8. External Lists – This is definitely near the top of my list. Being able to pull in a SQL table or even exposing relational databases and put together a story of business connectivity services exposed as a SharePoint list and taking that data offline in SharePoint! Come on this is a HUGE SQL story for usability and accessibility and extensibility. Loved the contacts BCS demos (see more on Channel 9) I’ve seen on SQL database editing with full CRUD (Create Read Update Delete) operations in SharePoint, as well as editing them in Outlook and taking them offline in Outlook in 2010 or in SharePoint workspace. The ability to create these external connections doesn’t take an expensive tool and can easily be created in free SharePoint designer 2010 or Visual Studio 2010. Add on the power of SQL 2008 R2 and powerpivot and now you have the most scalable lists of any app out there with serious performance improvements. 9. Cleaner SQL Backup and Recovery Story – While the backup story of the SSP gets cleaner with having a more robust backup for SQL databases right out of the box and a less messy Index and Search backup, the configuration based backup. You don’t need to do anything special to get config based backup. The configuration of your services is stored in the backup. So when you get ready to do your restore it will actually ask you. It will ask Do you want A) same or B) new configuration during restore. Pretty wild. Definitely simplifies the story. More info on the SharePoint Solutions Help blog on SharePoint 2010 Configuration Restore capability. 10.Resource Governor for SQL & SharePoint – The new schema of making databases specialized lends itself more to the idea of running resource governor to allow you to be smarter about resources. The new logging and usage databases are good candidates. On the SharePoint side it’s the resource governor of managing resources from a code and performance perspective which will ultimately save your front end web servers and save your backend, literally! Refer to the throttling link above on more resource management features as well. 1/31/2010Recent Posts, Article, Whitepapers Drill Down into SharePoint 2010 Upgrade Insights Can you tell I’ve been focused on SharePoint 2010 upgrade lately? I wanted to recap some of those posts for easy access. I’m doing some lectures/sessions at SPTechCon in a 2 part series on SharePoint 2010 Upgrade and needed to gather together some of the the top resources. I’ve put a list of my recently authored blog posts below. As well I’ve gathered some resources across TechNet and in the community and posted those on an updated post on my old blog in a post titled “SharePoint 2010 Upgrade Key Resources” Blogs: SharePoint 2010 Upgrade Resources on SharePointJoel.com Slides: Preparing for Upgrade to SharePoint 2010 Today – Slide deck on guidance to prepare today for upgrade Article: Preparing for SharePoint 2010 Upgrade Today – Key thoughts and talking points around planning for Upgrade to SharePoint 2010 SharePoint 2010 Upgrade Resource Centers Just in case you were wondering… Yes, there’s more. 1/28/2010While I agree consumer social media at work can be a conflict of interest for a business, but understanding the networking and valid business reasons of building the culture leveraging the new capabilities and transforming the business to better compete and stay on top of what the customer cares about… Do you want to block that too? I do NOT agree that blind firewall rules to block both blogs and micro blogging should simply be created to put up an invisible wall. Business departments across most companies will have valid BUSINESS reasons to use social media outlets and have legitimate reasons to participate in the networking or to data mine. In response to Christian Buckley’s Top 10 Reasons Your CIO Blocks Social Media I put together 10 Reasons your CIO should NOT block Social Networking sites. Should corporations be leary? Of course they should, but at the same time, they should understand both the upside and the downside, and not only that, understand what might happen even more uncontrolled if they block it anyway. The CIO is an important figure head, and the business is looking for serious guidance on technology. Is your CIO a social media evangelist? Nigel Fenwick of Forrester thinks he should be. So do I. Social Networking is coming to your enterprise, is your business ready for it? Here’s 10 reasons your CIO should not block Social Networking. 1. Innovation and Idea Incubation – With the Rapid pace of today’s environment the future of your company may depend on it. It’s where both industry and new ideas are being born. 2. Real customer interaction happens – whether or not you are there, your customers are there and they are either praising or bashing your product and looking to engage. Is someone there set to monitor it… at a minimum? 3. Social Networking – Think of the networking. Getting like people together. Your engineers and specialists need to be connecting with their industry and sharing ideas. 4. If you think you can block it with a proxy or firewall rule it will happen anyway – This is the 21st century and people have mobile devices and will likely be on it at home, which you can’t block anyway. How much better would it be if you could use the positive energy and at least coach the people on how to use it properly including time management. 5. Training and Expertise – Business is transforming at light speed. By leveraging what is happening your business can be taught to be more agile. You’d be surprised how much the ideas of the consumer social web translate into the enterprise social web. 6. It’s not going away – while it will most likely be transformed, social networking is advantageous for those that take advantage of it. As a platform for a CIO, it becomes a great way to become more approachable and scalable even more human. Your ability to connect and network with your employees can be appreciated as you share your thoughts on a blog and make an accessible profile. 7. Data Mining – Stop looking at Twitter as noise and people just chatting about what they ate for lunch, and do some searching, some topic trending, and build some charts. Look at Topsy search on your favorite terms and be blown away. 8. Community – When you start to look at what you could be building by simply catering to the needs of your product community. There’s a lot of reasons you want a community. Not just the obvious feedback, and research, but less understood changes in the heartbeat of your market. Awareness of trends, market awareness, and news. 9. A major shift has happened… The Web has transformed – Do your IT, Marketing, and HR departments and product business units get it? Do they understand how they can take advantage of it? There’s a competitive edge, is it understood? 10. Your Departments have reasons that will not be well understood by IT – HR has recruiting needs, Marketing has research, R&D has research, PR has some tracking they need to do. If it’s all blocked, they will find other ways of doing it, or live in ignorance, or pay someone else to gather and trust someone else's data. Oh, and as well, these are the obvious uses. There are likely way more reasons that each department could find that either research or engaging their customers or clients ads real business value that would be extremely expensive and slow by other means. All this social networking… With SharePoint 2010, will your company know how to leverage the technology? The transformation in the consumer world will provide a shot in the arm to boost enterprise social platforms. Enterprise corporate *Governance* and company culture are going to be KEY to that success. While simply blocking it won’t solve the problem and in fact may exacerbate the problem, forcing employees to reach out in more untraceable manner. Policies, practices are really the only things ultimately that you can do. Blocking it won’t solve your concerns, it will make your employees feel like you are out of touch, and old fashioned… and despite whether you get it or not, they’ll think you don’t. As they update their status on their mobile phone… my CIO is out of touch! There are more articles in this ongoing debate: Facebook, Twitter becoming business tools, but CIOs remain wary The CIO and Social Media: Social Police? 1/25/2010You hear about all the new features in SharePoint 2010, but what isn’t there? I bet you haven’t had many of those discussions yet. I think the news is better and I think the news is understandable. When you look through the methods in a list like this, it can sometimes look overwhelming, but compared to older versions, this is much much smaller than previous. There are a few notes to help you understand the deprecation from a developer perspective first, and then will discuss the features in the product. Files listed on Code.MSDN. Useful for troubleshooting and being aware of how the API types and methods have changed. SPS2010NewlyDeprecated.txt – A fairly short list of depreciated types and methods no longer supported in SharePoint 2010. SPS2010Deprecated.txt documentation, 528K, uploaded Oct 28 2009 - 573 downloads OfficeSharePointServer2007Deprecated.txt – to understand the API types and methods that were first deprecated in SharePoint 2007 including recommendations on many of them Depreciated Features to be aware of in SharePoint 2010: 1. Performance Point as a separate product – Now baked in/Included in SharePoint 2010 Enterprise. 2. My Site Host Redesign – No visual upgrade for my sites. The my sites really get a new look and feel… this is a good thing… really! Also be aware the Project Web Access as applicable doesn’t support visual upgrade (related to Project 2007 on SharePoint 2007 farm). I think most people didn’t customize this. 3. Side by Side Installation/Gradual Upgrade – No install of both binaries on the system, but you’ll be pleased that SharePoint 2010 ships with both 2007 and 2010 UI 4. SSP Admin UI – Any work you did on either the SSP Admin Site Collection or the Central Admin site has changed. 5. Central Admin - Admin task list and other lists that you might have created… You should move this to another site collection prior to upgrade. 6. Reporting Server webparts – New Design. Should work with an in place upgrade. 7. Deprecated Template - SSP Admin Site 8. Deprecated Template Great Plains (STSPKPL) – Template in 2003 plus pack. Use the Great Plains Integration Manager for richer SharePoint Integration with GP. 9. Features – PortalLayouts (Legacy) 10. Missing Assemblies - Web parts (STSPKPL) – This was also the case for 2003 to 2007. Shouldn’t be very common. If you were at the SharePoint Conference, there was a session on Upgrading Code to SharePoint 2010 as well Sean Livingston mentioned some of this in his advanced upgrade Let me know if I’m missing anything… 1/21/2010I’m happy to announce the public beta of Quest Site Administrator for SharePoint Reports On Demand, a new free SharePoint product that provides the ability to get reports without installing anything on the server. This new reporting functionality requires simply a simple download, a client install on Windows XP, Vista or Windows 7. You put your URL or URLs in the tool for reporting. If you are a site collection admin you can get recursive storage reports which include sizes of the various webs, including file type reports, and other reports to help understand your content storage. These reports leverage the Cloud, Microsoft’s Azure to make updates to the client software. The product is designed to be installed with no install on the server. The calls are from the client to the web services of the site. Hence the SharePoint web services are required, but they are on by default. These Silverlight reports and the update with Azure provides a platform of reporting that can continue to grow. “Site Administrator Reports OnDemand for SharePoint provides free overview reports for an unlimited number of SharePoint sites. The information in these reports allows you to assess the scope of the site you’re reviewing, understand how it is being used, and determine site storage metrics.” Get it at Quest.com/ReportsforSharePoint – No registration required! If you need any help or have any problems, there is a Reports for SharePoint forum where you can get help either from the community or from the team. 
Who can use it? SharePoint Server Admins, Site Collection Admins, even Site Administrators. All you really have to have is rights to get data out of the web services, so a viewer could ultimately get reports. Is it designed for Enterprises or as a Cloud? It can be used for either SharePoint Online or On premise, in your own environment. Will I eventually have to pay for it? No, the design is to keep it free as a fully working sample of what can be done. Quest Site Administrator for SharePoint for example has a lot more functionality and will remain the pay for version. What techologies are used for the reports? Silverlight 3 What browser and what SharePoint platform was this designed for? It was designed to be cross browser with support for SharePoint 2007, SharePoint 2010. It does require the Silverlight browser plug in. Where are the reports stored? Stored in client silverlight storage cache Does it scale, given this is designed against web services? It was tested with 100,000 and we’ll truncate the report at 10,000 items in a given list or library What is the main purpose of these reports? identifying large documents, understanding your storage better including files by type, understanding storage in versions Troubleshooting – Requires .NET 3.51 SP1, local admin access to your box, access to the SharePoint servers with Windows accounts (doesn’t work with Forms) most issues are related to not having either the right access locally or with access to the servers or authentication methods not being supported. 1/20/2010
SP2 had a number of enhancements designed around making it easier to upgrade to SharePoint 2010. One such new feature that wasn’t popularized is your ability to find and remove corrupted, broken sites known as orphans.
As I’ve mentioned in the past, with SharePoint being split across multiple databases, it is quite possible that a site could be in either the content database OR the config database.
With service pack 2 there’s a new way to find orphaned sites. In addition to running STSADM –o PreUpgradeCheck, one of my favorite tools along with TEST-SPContentDatabase with SharePoint 2010 you can identify these messed up corrupted sites.
You’ll run enumallwebs (STSADM reference on TechNet) to identify the orphaned sites, and use deletesite (STSADM reference on TechNet) to remove them. Note this command is available in SP2, and was enhanced in the post SP2 October Cumulative update. You really should consider that October update as the most significant release with upgrades since SP2, but also extremely relevant in getting ready for upgrade to SharePoint 2010. Look for the new –force across a few different commands. You don’t have to use the –force the first time around. Use it when things error out due to corruption.
Identifying Site Collections and Sites
Both Identify the sites missing from the Site Map and identify the GUID so it can be deleted.
STSADM –o EnumAllWebs
Syntax
stsadm -o enumallwebs
-databasename <database name>
[-databaseserver <database server name>]
With STSADM –o EnumAllWebs you can get a list of all of the sites. It may not seem very readable, but throw it into Excel or your favorite text editor or XML editor and clean it up. Once you’ve got it in the editor you’re going to look at the data where InSiteMap="False". Essentially what you’re looking at is a problem. If it’s not in the site map then it doesn’t think it really fully exists.
Delete Corrupted Sites
Syntax from TechNet
For deleting a live site collection:
stsadm -o deletesite
-url <URL name>
[-deleteadaccounts {True | False}]
[-gradualdelete]
For deleting a orphaned site collection:
-force
[-gradualdelete]
-siteid <site ID> (ID is listed in an output from EnumAllWebs)
-databasename <database name>
-databaseserver <database server name>
For Deleting Corruption
“The databaserepair operation can detect and repair database corruption for only the following types of orphaned items in a content database:
-
A Windows SharePoint Services Web site that does not have a parent Windows SharePoint Services Web site
-
A subweb that does not have a parent Windows SharePoint Services Web site
-
A list that does not have a parent Windows SharePoint Services Web site
-
A document that does not have a parent document library
-
A list item that does not have a parent list
-
A Web page that does not have a parent Windows SharePoint Services Web site”
Syntax
stsadm -o DatabaseRepair [-deletecorruption]
stsadm -o databaserepair
-url <url name>
-databasename <database name>
[-deletecorruption]
Delete Corrupted Webs
Syntax from TechNet
For deleting a live site within a site collection:
stsadm -o deleteweb
-url <URL name>
For deleting a site within an orphaned site collection or an orphaned site within a site collection:
-force
-webid <Web ID> (ID Comes from EnumAllWebs)
-databasename <database name>
-databaseserver <database server name>
The real useful feature is the new –force parameter. Essentially now you have the ability to add the –force when adding the URL of one of these orphaned sites. Before you’d simply get an error if you were trying to delete one of these sites. If you plan to use the –force parameter with your EnumAllWebs command you will need to provide the GUID which is the site id or web id. By simply looking at the XML output you can tell wether you need to run web or site command.
<Sites Count="2">
<Site Id="e2a23r5h-83c9-43t6-8wcf-3fesasefdsd234"
OwnerLogin="Domain\username" InSiteMap="False">
<Webs Count="1">
<Web Id="0ae3237b-b349-4693-bd67-dsefcsabb36"
Url="/sites/site_name" LanguageId="1033" TemplateName="STS#3" TemplateId="1" />
</Webs>
</Site>
<Site Id="fsdf32c4b8-79d9-32f6-87bf-4acreadcr3623"
OwnerLogin="Domain\username" InSiteMap="True">
<Webs Count="1">
<Web Id="234c8e7b-c456-4693-bd67-efdegd3bf466"
Url="/" LanguageId="1033" TemplateName="STS#0" TemplateId="1" />
</Webs>
</Site>
</Sites>
If you find the results from enumallwebs too overwhelming you can trim down what you’re looking at to a specific database. You can pass in the –database parameter and provide a specific database. For example: stsadm -o enumallwebs -databasename sharepoint_content_db
DON’T FORGET TO BACKUP YOUR DATABASES FIRST!
Here’s the KB that refers to the old ways of cleaning up orphaned sites
Repair content databases in Windows SharePoint Services http://support.microsoft.com/kb/918744
If anyone ever tells you to simply disconnect and reconnect your databases, you need to be very serious about what that means. It sounds very simple, and in simple environments that might be ok. In other environments you’ll find there’s a lot of layers of configuration around databases and by simply disconnecting and reconnecting you’ll lose the association with alerts and other services, essentially much of the configuration in the SSP that relates to those sites loses it’s way.
If you’re seeing a lot of corruption you should refer to this post on how SharePoint site corruption is created to see if you can follow better practices to avoid creating corruption. I’ve been pleased to see some in the community rally to help our those in Haiti. I’ve followed a few conversations on Twitter where friends such as Molly Hightower a friend of @klseargant was killed. The first day it happened I blew it off before I really knew the details. As soon as I realized the scale of it, and saw the faces of the people I looked at twitter and saw a bunch of posts with links, a lot on the text message to send $10 to Red Cross. It was very easy to do. One way I can help is simply bringing awareness of the efforts of those in the community putting in to help us understand the impact. An in the case of Dux who has setup a fund, I’d like to help in one way I can. Jose noticed there wasn’t many SharePoint tweets about Haiti. I think many of us don’t know what to say, and retweeting the noise of the photos and vidoes which are available on pretty much any news source might not help. I’d suggest that the community does have heart and is thinking about it. A lot. If you haven’t taken the time to contribute, I highly encourage you to, to your favorite charity. Not only will you feel good, you’ll help make this world a better place. There’s more than one angle, you’d find many of these organizations use SharePoint for mobilization, but we’ll save that for another post. Molly Hightower – Victim of the earthquake working in an orphanage. Read her blog about life in Haiti and the work she was doing and/or watch the video about her situation in Haiti and be touched. Inspiring, tragic and heartbreaking lk to blog by Molly Hightower, volunteer worker who died in Haiti: http://bit.ly/5pBWoX ryrivard: Molly Mackenzie Hightower, who died in the rubble in #Haiti, measured the year in minutes. Shldn't we all? http://mollyinhaiti.blogspot.com See a memorial video on KVPtv: @klsargent: friend of the family, Molly Hightower, was found in Haiti & she didn't make it. please support relief efforts there in any way you can!!! @klsargent Sorry to hear about your friend in Hati. I'm going to do the text 4 $10 Red Cross thing in Molly's memory Simple to help Haiti: Text HAITI to 90999. By doing this, you donate $10 to Red Cross 4 Haiti http://bit.ly/7lGqmX > Done Chicago #SharePoint peeps: Help Haiti relief efforts #ChiHelpsHaiti 1/21 @EnglishChicago 6-9pm RSVP http://bit.ly/7LcQ6F “There are two ways in which #ChiHelpsHaiti is assisting Chicagoans in donating to Haitian relief: a fundraiser will be held on Thursday, January 21st from 6:00 pm to 9:00 pm at English Bar, in Chicago's River North neighborhood; and between January 18th and January 24th, participating restaurants around the Chicago area will enable patrons to add a donation of $1 (or more) to their checks to support the cause.” meetdux.com - Operation ShareLove: Support Haiti Earthquake Victims meetdux: Working on a SharePoint fund drive to support Haiti relief efforts #ShareLove In an email from Dux Raymond Sy… “I hope you are doing well and enjoying 2010 so far. I am reaching out to you hoping that you can help spread the word about “Operation ShareLove: Haiti” fund drive that I just launched: http://bit.ly/8DWuUG It would be great if you can spend some time spreading the word by blogging, tweeting, facebooking (is this a verb?) about the fund drive. To make it easier, you can copy and paste the following text: "Operation ShareLove: Support Haiti Earthquake Victms & Win! http://bit.ly/8DWuUG"
I really appreciate any help you can provide and I believe we can truly make a difference as a community.
Let me know if you have any questions” @joseamorales @tweetsharepoint Have you tweeted about helping to Haiti? ShaePOint community can help guys! Dear #Sharepoint community members lets spread the word to help the people in Haiti. We can do it and please don't let it for later. @GuyKawasaki How about some remembering that is important to help Haiti? Guy you are a leader... stop with bs. GuyKawasaki: @joseamorales Yeah, we haven't done much: http://adjix.com/rmfi\ GuyKawasaki: 20+ ways to help Haiti http://om.ly/eHXv Ariel Gabriel Sobrino Haiti Links Blog Post in Spanish on SharePoint Evolucion 1/12/2010One of the greatest enhancements to upgrade to SharePoint 2010 is the work that was done in PreUpgradeCheck and in the powershell commandlets of Test-SPContentDatabase and Upgrade-SPContentDatabase. Test-SPContent Database is very similar to the stsadm command PreUpgradeCheck, but it works with both 2007 and 2010. This command can be pointed at a database that isn’t part of the farm! That’s the coolest. If you want to see how a content database would fare in a given farm you can run the Test-SPContentDatabase command and get information on issues that would impact the farm you’re importing it into. Is the database compatible? Does it have all the assemblies installed, all the relevant features and solutions all set? Well run the commandlet to find out. Many will be building new SharePoint 2010 farms and wanting to start clean from a hardware and software perspective. I do expect this to be the most common scenario. With just options of in-place upgrade or database attach, you’ll find this command to be the key. If you only run preupgradecheck (more resources below) on the farm you’re moving from, you’re missing important steps. Preparing for Upgrade with Powershell Test-SPContentDatabase Test-SPContentDatabase -Name -WebApplication [-AssignmentCollection ] [-DatabaseCredentials ] [-ServerInstance ] Test-SPContentDatabase –name SPContentDatabase –WebApplication http://test Here’s an example of a sample Test-SPContentDatabase 
In the example above, you can see examples of missing files, with a note to remedy by installing the missing feature with a path to the missing .dwp in this example. Also note that it shows if this will block the database attach upgrade. Pretty cool. If you were to compare PreUpgradeCheck and Test-SPContentDatabase you’d see the output and reporting is very different, but they do compliment each other. For example with database attach at a minimum you should run them both. On your source you should be running preupgradecheck and in the destination you should be running test-spcontentdatabase. Source (Your 2007 farm): PreUpgradeCheck will tell you what is broken or missing in the source. Destination (Your Clean 2010 Farm): Test-SPContentDatabase will tell you what you’ve missed in setting up your 2010 farm. There’s more information on TechNet on actually running the Test-SPContent database command. Here’s a couple of quotes on actual usage. Note, this is beta content and it may change. There’s some hidden gems of tips in these paragraphs below. “Before you add the content databases to the Web applications, you can use a Windows PowerShell cmdlet to verify that you have all the custom components that you need for that database. At the command prompt, run the following cmdlet: Test-SPContentDatabase –Name <database name> -WebApplication <URL> [-ServerInstance <ServerInstanceName>] [-DatabaseCredentials <Domain\username>] When you add the content databases, be sure that the root site for the Web application is included in the first content database that you add. In other words, before you continue, examine the root of the Web application in the original server farm to determine the first site collection. After you add the database that contains the root site, you can add the other content databases for the Web application in any order. You do not have to create any site collections to store the content before you add the database; this process creates the site collections for you. Be sure that you do not add any new site collections until you have restored all the content databases. You must use the Stsadm command-line tool to add a content database to a Web application. Using the SharePoint Central Administration pages to attach a content database is not supported for upgrading.” emphasis added be me. Database Attach with STSADM AddContentDb “To add a content database to a Web application, you must use the addcontentdb operation.” (Emphasis added) Simple: stsadm -o addcontentdb -url <URL> -databasename <database name> With Options: stsadm.exe -o addcontentdb -url <URL name> [-assignnewdatabaseid] [-clearchangelog] -databasename <database name> [-databaseserver <database server name>] [-databaseuser <database username>] [-databasepassword <database password>] [-sitewarning <site warning count>] [-sitemax <site max count>] If you are actually running a database attach upgrade please note that some of these options on the upgrade are added for specific reasons related to troubleshooting issues, such as clearing the change log and assigning a new database id because it’s already taken. The powershell command Upgrade-SPContentDatabase can be used to resume a database attach failure. So you’d use the test-spcontentdatabase to check for missing, and use stsadm –o addcontentdb to add the content database, and use upgrade-spcontentdatabase to resume upgrades with issues. After Infra Update each content database IDs are retained when you restore or reattach the database by using built-in tools. “Default change log retention behavior when using built-in tools is as follows: -
The change logs for all databases are retained when you restore a farm. -
The change log for a content database is retained when you reattach the database. -
The change log for a content database is NOT retained when you restore just the content database. Clearing the change log forces Search to run a full crawl on that database so that the index no longer references items that do not exist.” Verify Status with Upgrade Logs and Upgrade Status Page There are many places and ways you can check upgrade status. You can use the Upgrade Status page in Central Administration to check the status of upgrade on your site collections and then check the upgrade log, and run STSADM. - STSADM -o localupgradestatus (To see if any sites were missed or skipped for upgrade, be sure to run it on all WFEs.)
- Upgrade Status page – From Central Administration, click Upgrade and Migration then Check upgrade status.
- To open the upgrade and err log files - %COMMONPROGRAMFILES%\Microsoft Shared\web server extensions\14\LOGS. Note there are logs for each session.
Recap with Steps for 2007 to 2010 database attach upgrade based on the information in this post: Of Course.. Acquire hardware, run lots of backups along the way, and build out the 2010 farm with all the best practices, run it all on a test environment, etc… Also note this is back of the napkin, obviously you need to add you own additional steps of communication, validation, and any other additional operational steps, etc… 1. Run PreUpgradeCheck on 2007 farm 2. Fix and address issues 3. Run PreUpgradeCheck to verify all issues addressed 4. Run Test-SpContentDatabase 5. Fix issues and make planning choices appropriately 6. Re-Rerun Test-SpContentDatabase 7. Upgrade the Service Apps and verify they are all good, then Run stsadm –o addcontentdatabase to add the relevant databases to the relevant web apps 8. Check the upgrade status both in central admin and in stsadm –o localupgradestatus on each SharePoint server and check the logs to verify the upgrade process itself was successful. 9. Re-Run Test-SPContentDatabase and verify database upgrade of sites was successful 10. Troubleshoot any issues then run Upgrade-SPContentDatabase if any databases need to resume upgrade 10. Visual verification --- Ready to start visual upgrade Powershell Commands useful around Upgrade Test-SpContentDatabase – discussed in this post Upgrade-SPContentDatabase – to resume failed upgrades Upgrade-SPEnterpriseSearchServiceApplication – Upgrade Search Upgrade-SPSingleSignOnDatabase – Upgrade SSO STSADM Commands useful around preparing for Upgrade stsadm -o ExportIPFSAdminObjects stsadm -o MergeContentDB stsadm -o EnumAllWebs stsadm -o DatabaseRepair [-deletecorruption] stsadm -o DeleteSite [-force] [-gradualdelete] stsadm -o DeleteWeb [-force] stsadm -o ForceDeleteList stsadm -o VariationsFixupTool stsadm –o Upgrade (Used for build to build upgrades, including service packs.) More on TechNet Use a trial upgrade to find potential issues (SharePoint Server 2010) Run the pre-upgrade checker (SharePoint Server 2010) Prepare the new SharePoint Server 2010 environment for a database attach upgrade PreUpgradeCheck Additional Resources from the Community: Todd Klindt Using Powershell to Control Visual Upgrade Gary LaPoint STSADM and Powershell in SharePoint 2010 Eric Kraus - Automating SharePoint 2010 Install with Powershell |
|