SharePoint 2013 Storage improvements and RBS Considerations

As we know one of the SharePoint’s key capability is Document libraries and the files within these libraries internally stored in content databases as BLOBs along with metadata.

RBS is used to extend the storage of file outside of SharePoint to optimize the overall storage cost. With SharePoint 2013, there are architectural changes in how SQL stores files and also how WFEs deal with those files.  This new capability made available is called “Shredded storage”.

So it is important to make proper design decision to leverage SP 2013’s shredded storage before considering RBS.

Let us start dive into the details to understand these concepts better.

SharePoint 2010 storage and retrieval

With SharePoint 2010, when the end user updates the document, only the changes used to be send back over MS-FSSHHTTP and the Web Server used to load the complete BLOB from database and updates the parts in it and stores the updated BLOB into Database again.

 SharePoint 2010 way of document changes are explained below

 

The problem with the above approach is there is overload on WFEs even when a portion of the document changed.

Now we will see how Shredded storage with SharePoint 2013 addressed the above problem

Shredded Storage:

Shredded Storage is a new data platform improvement provided with SharePoint 2013 related to the management of large binary objects.

The files are broken into pieces and stored as multiple BLOBs instead of a single BLOB. On update only the related chunks will get updated on SQL.

 

 

Shredded Storage in SharePoint 2013 optimizes I/O and provides smoother I/O patterns when compared to prior versions of SharePoint Products, and it provides storage cost benefits under conditions where historical versioning is widely used and implemented.

By default, Shredded Storage partitions data across a 64 KB boundary based on the FileWriteChunkSize property, which is configurable up to a maximum partition size of 2 GB. 2 GB is also representative of the maximum file size supported by SharePoint 2013.

RBS and Shredded Storage

Remote Blob Storage (RBS) was introduced in SharePoint 2010 and continues to exist in 2013. RBS is best used to push relatively large files from being contained directly in the SharePoint content database(s) out into an actual file system.

The Blob threshold can be adjusted by “MinimumBlobStorageSize “setting which controls minimum size of files to move to RBS by the Management Shell on SharePoint server.

Let us think about both RBS and Shredded Storage

Ø  RBS works best with larger blobs.

Ø  Shredded Storage slices larger blobs into a lot of very small blobs.

So the guidance is to consider RBS only when there are large files which are mainly existing for ready only purpose ex: Record Library and Archive Library.  Evaluate FileWriteChunkSize and MinimumBlobStorageSize s properly to avoid unexpected results.

Routing the Files in SharePoint

In the Document management space one of the requirements is routing the documents automatically into appropriate folders or sub folders inside a document library.
This article explains about the Out of box feature that is provided to meet the above requirement, the limitation of SharePoint 2010 and the Custom solution which meets the requirement

Introduction

In the Document management space one of the requirements is routing the documents automatically into appropriate folders or sub folders inside a document library.

This article explains about the Out of box feature that is provided to meet the above requirement, the limitation of SharePoint 2010 and the Custom solution which meets the requirement

OOTB SharePoint Option:

In SharePoint 2010 there is a feature called Content Organizer rule which addresses the above need through its attributes called Target location > Destination.

See those attributes in the below content organizer rule

 

 

Limitation with the content organizer rule routing

Whereas there is a limitation with these attributes in terms of creating hierarchical folders. For example if the requirement is to create multiple levels of folders/subfolders inside the target location based on the selected metadata it is not possible with the OOTB content organizer rule. It can only create one level below the target location.

How to achieve this functionality through Custom Router?

We can leverage Custom router to create the hierarchical folders and route the document appropriately to the corresponding folder.

Custom router is the custom class library that implements ICustomRouter interface and this interface has one method called OnSubmitFil which needs to be implemented by the custom router’s class.

Implementing Custom router:

Routing can be customized by implementing an Interface called “ICustomRouter” which has only one method named “OnSubmitFile”. This code will be called by Content Organizer internally.

Here is the signature of OnSubmitFile

CustomRouterResult OnSubmitFile(

         EcmDocumentRoutingWeb contentOrganizerWeb,

         string recordSeries,

         string userName,

         Stream fileContent,

         RecordsRepositoryProperty[] properties,

         SPFolder finalFolder,

         ref string resultDetails

)

Please refer http://msdn.microsoft.com/en-us/library/ee561223 for more details on this interface.

Perform the below steps to implement this method to meet our requirement.

1.       Retrieve the properties from the parameter called properties.

2.       Push all the values into a Hashtable

3.       Read the required properties based on which we need to create the hierarchical folder structure.

4.       See whether this folder structure already exists.

5.       Create the folder structure if it does not exist.

6.       Move the file into the folder.

See the sample code below

public class MyCustomRouter : ICustomRouter

    {

       

      CustomRouterResult ICustomRouter.OnSubmitFile(

                EcmDocumentRoutingWeb contentOrganizerWeb,

                string recordSeries, // Content type name

                string userName,

                Stream fileContent,

                Microsoft.SharePoint.RecordsRepositoryProperty[] properties,

                SPFolder finalFolder,

                ref string resultDetails)

        {

         

          

  string fileContentString = string.Empty;

  StreamReader sr = new StreamReader(fileContent);

  {

    fileContentString = sr.ReadToEnd();

  }

  using (SPSite site = new SPSite(contentOrganizerWeb.DropOffZoneUrl))

    {

        SPWeb web = site.OpenWeb();

        // User creating the file

      

        // Create a Hashtable of properties which forms the metadata for the file

        Hashtable fileProperties = EcmDocumentRouter.GetHashtableForRecordsReposi       toryProperties(properties, recordSeries);

       

EcmDocumentRouter.SaveFileToFinalLocation(contentOrganizerWeb,

    finalFolder,

    finalStm,

    fileName,

    "",

    fileProperties,

    submittingUser,

    true /*override versioning settings on the content organizer and create a new file*/, "");

        }

        resultDetails = "File was inspected and sensitive data was found. File has been saved with a custom name.";

        return CustomRouterResult.SuccessCancelFurtherProcessing;

    }

}

        }

    }

Now the custom router is ready to use. How this router will be attached to the Content Organizer?

Attaching Custom router to the Content Organizer rule:

The custom code mentioned in the above section need to compile into an assembly and sign the assembly with the key. Now the assembly is ready to add into the GAC where SharePoint is running.

Add the assembly into GAC and it is ready to use inside the SharePoint.

Modify the Content Organizer rule:

Go to the Content Organizer rule to which we want to attach the custom router. Edit properties and go to Custom router field and select our newly created Custom router assembly over there.

Click on OK will save the Content Organizer rule and Custom router will route the incoming documents based on the logic written in Custom Router

Conclusion

This article explains the power of Custom router and how can you route the files automatically based on the property values.

Reference

http://msdn.microsoft.com/en-us/library/ee561223

Upgrade and Migration for SharePoint 2010

This document explains the SharePoint 2010 migration approach and the important points that we should consider during and after the migration.

Upgrade approach:

There are two basic approaches to upgrade: in-place and database attach. With in-place approach we can install SharePoint Server 2010 on the same hardware of MOSS 2007 and with Database attach approach we can upgrade the content on a different farm.

Since we wanted to setup new multi-farm for SharePoint, we have chosen the second approach.

Database attach upgrade:

With this approach the content for the environment will be upgraded on a separate farm. The result is that there is no need to upgrade any of the services or farm settings. Upgrade the databases in any order and upgrade several databases at the same time. While each database is being upgraded, the content in that database is not available to users.

We can upgrade multiple content databases at the same time, which results in faster upgrade times overall than an in-place upgrade. You can use a database attach upgrade to combine multiple farms into one farm.

Whereas the server and farm settings are not upgraded. We must manually transfer settings that we want to preserve from the old farm to the new farm. Any customizations must also be transferred to the new farm manually. Any missing customizations may cause unintended losses of functionality or user experience issues.

Major changes should consider during or after upgrade:

1.     Search:

In SharePoint Server 2010, the Search service uses three databases:

1. Search administration database (new): contains Search administration settings that were stored in the SSP database in Office SharePoint Server 2007.

2. Search Service Crawl database (new): contains crawl history information that was stored in the SSP database in SharePoint Server 2007.

3. Search metadata database (reused Search database): contains the metadata for search.

2.     User Profiles and Taxonomy:

Two services are now used for user profiles and taxonomy information: the User Profile service and the Managed Metadata service.

During a database attach upgrade, user profile and taxonomy data from the SSP database is upgraded when the SSP database is attached, but the database is not copied and renamed. You can copy the taxonomy data into a Taxonomy database for use by the Managed Metadata service after upgrade is complete.

3.     Master Pages:

A site owner can now apply branding to their site, independent of other sites, and administrators can specify whether the system pages in the _Layouts folder are rendered by using the site master pages provided by site owners or by default master pages available across the system. Also, it is possible to use Windows PowerShell to specify a customer master page to system error pages, login pages, confirmation pages, and other non-site-specific pages.

4.     HTML Markup changes in rendering:

There is a change in HTML markup generated for the SharePoint from 2007 to 2010 version. For cross browser support they have replaced the tables with Div and ul, li etc. So, this is a very good thinking and change from SharePoint team which likes by all other browsers.

We need to make sure all our customized styles should be converted from Tables to Divs.

5.     Visual upgrade:

Once we are done with SharePoint site migration to SharePoint 2010, we will see the same layout and UI as previous site. The UI is not really migrated to the SharePoint 2010 UI [V4]. So, what to do to migrate UI too to SharePoint 2010 format?
Now, we can see a new item in the site actions menu named "Visual Upgrade" as shown below.

When we click on the Visual Upgrade option, then you go to a page we can see three options to preview how it looks like in v4.

1. Here, the option 1 means, display the same UI as the v3 UI. i.e. Your SharePoint 2007 UI.
2. Option 2 is, Preview the site in V4 mode, but, don't save it permanently. So that you have chance to take a look at the site in V4 mode and if you think UI is not well formatted then revert the changes back to V3.
3. Option 3 is, permanently change the UI to V4 mode. If you select this, then you can't change the UI back to V3 mode and the option "Visual Upgrade" under site actions will go away

6.     Assemblies/References:

 Any custom code assemblies need to now be assembled in Visual Studio as x64 (64-bit) or “any”, not as x86 (32-bit).  There is no scenario in SharePoint 2010 that would warrant a 32-bit assembly, as SharePoint 2010 is 64-bit only.  AssemblyBinding entries in the web.config file will use binding parameters in SharePoint 2010 to redirect references from the 12 hive to the 14 hive.  This will use <bindingRedirect oldversion=”12.0.0.0” newversion=”14.0.0.0”> to redirect any assembly IDs from 12 to 14.

7.     Content type hub:

SharePoint 2010 introduces a new concept called the 'Content Type Hub'. It is a centralized application where users can create different content types and publish them. As it would be defined as content type hub, every application could use it with a single configuration. The advantage is it would be centralized and could be shared among different applications.