Adds a ColdBootType so we can detect and work with that if we have our own custom callbacks

Add some green to 'stay' option - U4-9323

.gitignore

@@ -141,3 +141,4 @@ build/docs.zip
 build/ui-docs.zip
 build/csharp-docs.zip
 build/msbuild.log
+.vs/

README.md

@@ -1,8 +1,15 @@
 Umbraco CMS
 ===========
-Umbraco is a free open source Content Management System built on the ASP.NET platform.
+The friendliest, most flexible and fastest growing ASP.NET CMS used by more than 390,000 websites worldwide: [https://umbraco.com](https://umbraco.com)
+
+[![ScreenShot](vimeo.png)](https://vimeo.com/172382998/)
+
+## Umbraco CMS ##
+Umbraco is a free open source Content Management System built on the ASP.NET platform. Our mission is to help you deliver delightful digital experiences by making Umbraco friendly, simpler and social.
+
 
 ## Building Umbraco from source ##
+
 The easiest way to get started is to run `build/build.bat` which will build both the backoffice (also known as "Belle") and the Umbraco core. You can then easily start debugging from Visual Studio, or if you need to debug Belle you can run `grunt vs` in `src\Umbraco.Web.UI.Client`.
  
 If you're interested in making changes to Belle without running Visual Studio make sure to read the [Belle ReadMe file](src/Umbraco.Web.UI.Client/README.md). 
@@ -13,30 +20,33 @@ Note that you can always [download a nightly build](http://nightly.umbraco.org/?
 
 [![ScreenShot](http://umbraco.com/images/whatisumbraco.png)](https://umbraco.tv/videos/umbraco-v7/content-editor/basics/introduction/cms-explanation/)
 
-## Umbraco - the simple, flexible and friendly ASP.NET CMS ##
+## Umbraco - The Friendly CMS ##
 
-**More than 350,000 sites trust Umbraco** 
+For the first time on the Microsoft platform, there is a free user and developer friendly CMS that makes it quick and easy to create websites - or a breeze to build complex web applications. Umbraco has award-winning integration capabilities and supports ASP.NET MVC or Web Forms, including User and Custom Controls, out of the box. 
 
-For the first time on the Microsoft platform, there is a free user and developer friendly CMS that makes it quick and easy to create websites - or a breeze to build complex web applications. Umbraco has award-winning integration capabilities and supports ASP.NET MVC or Web Forms, including User and Custom Controls, out of the box. It's a developer's dream and your users will love it too. 
+Umbraco is not only loved by developers, but is a content editors dream. Enjoy intuitive editing tools, media management, responsive views and approval workflows to send your content live.
 
-Used by more than 350,000 active websites including [http://daviscup.com](http://daviscup.com), [http://heinz.com](http://heinz.com), [http://peugeot.com](http://peugeot.com), [http://www.hersheys.com/](http://www.hersheys.com/) and **The Official ASP.NET and IIS.NET website from Microsoft** ([http://asp.net](http://asp.net) / [http://iis.net](http://iis.net)), you can be sure that the technology is proven, stable and scales.  
+Used by more than 350,000 active websites including Carlsberg, Segway, Amazon and Heinz and **The Official ASP.NET and IIS.NET website from Microsoft** ([https://asp.net](https://asp.net) / [https://iis.net](https://iis.net)), you can be sure that the technology is proven, stable and scales. Backed by the team at Umbraco HQ, and supported by a dedicated community of over 200,000 craftspeople globally, you can trust that Umbraco is a safe choice and is here to stay. 
 
-To view more examples, please visit [http://umbraco.com/why-umbraco/#caseStudies](http://umbraco.com/why-umbraco/#caseStudies)
+To view more examples, please visit [https://umbraco.com/why-umbraco/#caseStudies](https://umbraco.com/why-umbraco/#caseStudies)
+
+## Why Open Source? ##
+As an Open Source platform, Umbraco is more than just a CMS. We are transparent with our roadmap for future versions, our incremental sprint planning notes are publicly accessible and community contributions and packages are available for all to use.
 
 ## Downloading ##
 
-The downloadable Umbraco releases live at [http://our.umbraco.org/download](http://our.umbraco.org/download).
+The downloadable Umbraco releases live at [https://our.umbraco.org/download](https://our.umbraco.org/download).
 
 ## Forums ##
 
-We have a forum running on [http://our.umbraco.org](http://our.umbraco.org). The discussions group on [Google Groups](https://groups.google.com/forum/#!forum/umbraco-dev) is for discussions on developing the core, and not on Umbraco-implementations or extensions in general. For those topics, please use [http://our.umbraco.org](http://our.umbraco.org).
+Peer-to-peer support is available 24/7 at the community forum on [https://our.umbraco.org](https://our.umbraco.org). 
 
 ## Contribute to Umbraco ##
 
-If you want to contribute back to Umbraco you should check out our [guide to contributing](http://our.umbraco.org/contribute).
+Umbraco is contribution focused and community driven. If you want to contribute back to Umbraco please check out our [guide to contributing](https://our.umbraco.org/contribute).
 
 ## Found a bug? ##
 
-Another way you can contribute to Umbraco is by providing issue reports. For information on how to submit an issue report refer to our [online guide for reporting issues](http://our.umbraco.org/contribute/report-an-issue-or-request-a-feature).
+Another way you can contribute to Umbraco is by providing issue reports. For information on how to submit an issue report refer to our [online guide for reporting issues](https://our.umbraco.org/contribute/report-an-issue-or-request-a-feature).
 
 To view existing issues, please visit [http://issues.umbraco.org](http://issues.umbraco.org).

build/NuSpecs/UmbracoCms.Core.nuspec

@@ -27,14 +27,14 @@
       <dependency id="SharpZipLib" version="[0.86.0, 1.0.0)" />
       <dependency id="MySql.Data" version="[6.9.8, 7.0.0)" />
       <dependency id="xmlrpcnet" version="[2.5.0, 3.0.0)" />
-      <dependency id="ClientDependency" version="[1.9.1, 2.0.0)" />
+      <dependency id="ClientDependency" version="[1.9.2, 2.0.0)" />
       <dependency id="ClientDependency-Mvc5" version="[1.8.0, 2.0.0)" />
 	  <!-- AutoMapper can not be updated due to: https://github.com/AutoMapper/AutoMapper/issues/373#issuecomment-127644405 -->
       <dependency id="AutoMapper" version="[3.3.1, 4.0.0)" />
       <dependency id="Newtonsoft.Json" version="[6.0.8, 10.0.0)" />
       <dependency id="Examine" version="[0.1.70, 1.0.0)" />
-      <dependency id="ImageProcessor" version="[2.4.5, 3.0.0)" />
-      <dependency id="ImageProcessor.Web" version="[4.6.6, 5.0.0)" />
+      <dependency id="ImageProcessor" version="[2.5.1, 3.0.0)" />
+      <dependency id="ImageProcessor.Web" version="[4.7.2, 5.0.0)" />
       <dependency id="semver" version="[1.1.2, 2.0.0)" />
       <dependency id="UrlRewritingNet" version="[2.0.7, 3.0.0)" />	  
 	  <!-- Markdown can not be updated due to: https://github.com/hey-red/markdownsharp/issues/71#issuecomment-233585487 -->
@@ -103,4 +103,4 @@
     <file src="..\_BuildOutput\bin\UmbracoExamine.pdb" target="lib" />
     <file src="..\..\src\UmbracoExamine\**\*.cs" exclude="..\..\src\**\TemporaryGeneratedFile*.cs" target="src\UmbracoExamine" />
   </files>
-</package>
+</package>

build/NuSpecs/UmbracoCms.nuspec

@@ -17,7 +17,8 @@
     <dependencies>
       <dependency id="UmbracoCms.Core" version="[$version$]" />
       <dependency id="Newtonsoft.Json" version="[6.0.8, 10.0.0)" />
-      <dependency id="Umbraco.ModelsBuilder" version="[3.0.4, 4.0.0)" />
+      <dependency id="Umbraco.ModelsBuilder" version="[3.0.5, 4.0.0)" />
+      <dependency id="ImageProcessor.Web.Config" version="[2.3.0, 3.0.0)" />	  
     </dependencies>
   </metadata>
   <files>
@@ -43,6 +44,7 @@
     <file src="tools\trees.config.install.xdt" target="Content\config\trees.config.install.xdt" />
     <file src="tools\umbracoSettings.config.install.xdt" target="Content\config\umbracoSettings.config.install.xdt" />
     <file src="tools\Views.Web.config.install.xdt" target="Views\Web.config.install.xdt" />
+    <file src="tools\processing.config.install.xdt" target="Content\Config\imageprocessor\processing.config.install.xdt" />
     <file src="build\**" target="build" />
   </files>
 </package>

build/NuSpecs/tools/Web.config.install.xdt

@@ -335,7 +335,7 @@
 
   <runtime>
     <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
-      <dependentAssembly>
+      <dependentAssembly xdt:Transform="Insert">
         <assemblyIdentity name="HtmlAgilityPack" publicKeyToken="bd319b19eaf3b43a" culture="neutral" />
         <bindingRedirect oldVersion="0.0.0.0-1.4.9.0" newVersion="1.4.9.0" />
       </dependentAssembly>

build/NuSpecs/tools/install.core.ps1

@@ -71,6 +71,7 @@ if ($project) {
 		if(Test-Path $umbracoBinFolder\ImageProcessor.dll) { Remove-Item $umbracoBinFolder\ImageProcessor.dll -Force -Confirm:$false }
 		if(Test-Path $umbracoBinFolder\ImageProcessor.Web.dll) { Remove-Item $umbracoBinFolder\ImageProcessor.Web.dll -Force -Confirm:$false }
 		if(Test-Path $umbracoBinFolder\Lucene.Net.dll) { Remove-Item $umbracoBinFolder\Lucene.Net.dll -Force -Confirm:$false }
+		if(Test-Path $umbracoBinFolder\Microsoft.IO.RecyclableMemoryStream.dll) { Remove-Item $umbracoBinFolder\Microsoft.IO.RecyclableMemoryStream.dll -Force -Confirm:$false }
 		if(Test-Path $umbracoBinFolder\Microsoft.AspNet.Identity.Core.dll) { Remove-Item $umbracoBinFolder\Microsoft.AspNet.Identity.Core.dll -Force -Confirm:$false }
 		if(Test-Path $umbracoBinFolder\Microsoft.AspNet.Identity.Owin.dll) { Remove-Item $umbracoBinFolder\Microsoft.AspNet.Identity.Owin.dll -Force -Confirm:$false }
 		if(Test-Path $umbracoBinFolder\Microsoft.CodeAnalysis.CSharp.dll) { Remove-Item $umbracoBinFolder\Microsoft.CodeAnalysis.CSharp.dll -Force -Confirm:$false }

build/NuSpecs/tools/processing.config.install.xdt

@@ -0,0 +1 @@
+<processing xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform" xdt:Transform="SetAttributes(preserveExifMetaData)" preserveExifMetaData="true" />

build/UmbracoVersion.txt

@@ -1,2 +1,2 @@
 # Usage: on line 2 put the release version, on line 3 put the version comment (example: beta)
-7.5.4
+7.5.7

src/SQLCE4Umbraco/SqlCE4Umbraco.csproj

@@ -49,13 +49,9 @@
     <Reference Include="System.Core" />
     <Reference Include="System.Data.SqlServerCe, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
       <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.dll</HintPath>
-      <SpecificVersion>False</SpecificVersion>
-      <Private>False</Private>
     </Reference>
     <Reference Include="System.Data.SqlServerCe.Entity, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
       <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.Entity.dll</HintPath>
-      <SpecificVersion>False</SpecificVersion>
-      <Private>False</Private>
     </Reference>
     <Reference Include="System.Xml.Linq" />
     <Reference Include="System.Data.DataSetExtensions" />

src/SQLCE4Umbraco/SqlCEInstaller.cs

@@ -21,15 +21,15 @@ namespace SqlCE4Umbraco
     public class SqlCEInstaller : DefaultInstallerUtility<SqlCEHelper>
     {
         #region Private Constants
-       
+
         /// <summary>The latest database version this installer supports.</summary>
         private const DatabaseVersion LatestVersionSupported = DatabaseVersion.Version4_8;
 
         /// <summary>The specifications to determine the database version.</summary>
         private static readonly VersionSpecs[] m_VersionSpecs = new VersionSpecs[] {
-					new VersionSpecs("SELECT CONSTRAINT_NAME FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS LEFT OUTER JOIN umbracoApp ON appAlias = appAlias WHERE CONSTRAINT_NAME = 'FK_umbracoUser2app_umbracoApp'", 0, DatabaseVersion.Version4_8), 
-					new VersionSpecs("SELECT id FROM umbracoNode WHERE id = -21", 1, DatabaseVersion.Version4_1),        
-					new VersionSpecs("SELECT action FROM umbracoAppTree",DatabaseVersion.Version4),
+                    new VersionSpecs("SELECT CONSTRAINT_NAME FROM INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS LEFT OUTER JOIN umbracoApp ON appAlias = appAlias WHERE CONSTRAINT_NAME = 'FK_umbracoUser2app_umbracoApp'", 0, DatabaseVersion.Version4_8),
+                    new VersionSpecs("SELECT id FROM umbracoNode WHERE id = -21", 1, DatabaseVersion.Version4_1),
+                    new VersionSpecs("SELECT action FROM umbracoAppTree",DatabaseVersion.Version4),
                     new VersionSpecs("SELECT description FROM cmsContentType",DatabaseVersion.Version3),
                     new VersionSpecs("SELECT id FROM sysobjects",DatabaseVersion.None) };
 
@@ -43,8 +43,9 @@ namespace SqlCE4Umbraco
         public override bool CanConnect
         {
             get
-            {                
-                SqlHelper.CreateEmptyDatabase();
+            {
+                using (var sqlHelper = SqlHelper)
+                    sqlHelper.CreateEmptyDatabase();
                 return base.CanConnect;
             }
         }
@@ -93,22 +94,22 @@ namespace SqlCE4Umbraco
 
         #region DefaultInstaller Members       
 
-		/// <summary>
-		/// Returns the sql to do a full install
-		/// </summary>
-		protected override string FullInstallSql
-		{
-			get { return string.Empty; }
-		}
+        /// <summary>
+        /// Returns the sql to do a full install
+        /// </summary>
+        protected override string FullInstallSql
+        {
+            get { return string.Empty; }
+        }
 
 
-		/// <summary>
-		/// Returns the sql to do an upgrade
-		/// </summary>
-		protected override string UpgradeSql
-		{
-			get { return string.Empty; }
-		}
+        /// <summary>
+        /// Returns the sql to do an upgrade
+        /// </summary>
+        protected override string UpgradeSql
+        {
+            get { return string.Empty; }
+        }
 
         // We need to override this as the default way of detection a db connection checks for systables that doesn't exist
         // in a CE db
@@ -123,8 +124,9 @@ namespace SqlCE4Umbraco
             // verify connection
             try
             {
-                if (SqlCeApplicationBlock.VerifyConnection(base.SqlHelper.ConnectionString))
-                    return DatabaseVersion.None;
+                using (var sqlHelper = SqlHelper)
+                    if (SqlCeApplicationBlock.VerifyConnection(sqlHelper.ConnectionString))
+                        return DatabaseVersion.None;
             }
             catch (Exception e)
             {

src/SQLCE4Umbraco/SqlCETableUtility.cs

@@ -37,13 +37,15 @@ namespace SqlCE4Umbraco
             ITable table = null;
 
             // get name in correct casing
-            name = SqlHelper.ExecuteScalar<string>("SELECT name FROM sys.tables WHERE name=@name",
-                                                          SqlHelper.CreateParameter("name", name));
+            using (var sqlHelper = SqlHelper)
+                name = sqlHelper.ExecuteScalar<string>("SELECT name FROM sys.tables WHERE name=@name",
+                                                          sqlHelper.CreateParameter("name", name));
             if (name != null)
             {
                 table = new DefaultTable(name);
 
-                using (IRecordsReader reader = SqlHelper.ExecuteReader(
+                using (var sqlHelper = SqlHelper)
+                using (IRecordsReader reader = sqlHelper.ExecuteReader(
                         @"SELECT c.name AS Name, st.name AS DataType, c.max_length, c.is_nullable, c.is_identity
                           FROM sys.tables AS t
                             JOIN sys.columns AS c ON t.object_id = c.object_id
@@ -51,7 +53,7 @@ namespace SqlCE4Umbraco
                             JOIN sys.types AS ty ON ty.user_type_id = c.user_type_id
                             JOIN sys.types st ON ty.system_type_id = st.user_type_id
                           WHERE t.name = @name
-                          ORDER BY c.column_id", SqlHelper.CreateParameter("name", name)))
+                          ORDER BY c.column_id", sqlHelper.CreateParameter("name", name)))
                 {
                     while (reader.Read())
                     {
@@ -112,7 +114,8 @@ namespace SqlCE4Umbraco
 
             // create query
             StringBuilder createTableQuery = new StringBuilder();
-            createTableQuery.AppendFormat("CREATE TABLE [{0}] (", SqlHelper.EscapeString(table.Name));
+            using (var sqlHelper = SqlHelper)
+                createTableQuery.AppendFormat("CREATE TABLE [{0}] (", sqlHelper.EscapeString(table.Name));
 
             // add fields
             while (hasNext)
@@ -136,7 +139,8 @@ namespace SqlCE4Umbraco
             // execute query
             try
             {
-                SqlHelper.ExecuteNonQuery(createTableQuery.ToString());
+                using (var sqlHelper = SqlHelper)
+                    sqlHelper.ExecuteNonQuery(createTableQuery.ToString());
             }
             catch (Exception executeException)
             {
@@ -154,13 +158,15 @@ namespace SqlCE4Umbraco
             Debug.Assert(table != null && field != null);
 
             StringBuilder addColumnQuery = new StringBuilder();
-            addColumnQuery.AppendFormat("ALTER TABLE [{0}] ADD [{1}] {2}",
-                                        SqlHelper.EscapeString(table.Name),
-                                        SqlHelper.EscapeString(field.Name),
-                                        SqlHelper.EscapeString(GetDatabaseType(field)));
+            using (var sqlHelper = SqlHelper)
+                addColumnQuery.AppendFormat("ALTER TABLE [{0}] ADD [{1}] {2}",
+                                        sqlHelper.EscapeString(table.Name),
+                                        sqlHelper.EscapeString(field.Name),
+                                        sqlHelper.EscapeString(GetDatabaseType(field)));
             try
             {
-                SqlHelper.ExecuteNonQuery(addColumnQuery.ToString());
+                using (var sqlHelper = SqlHelper)
+                    sqlHelper.ExecuteNonQuery(addColumnQuery.ToString());
             }
             catch (Exception executeException)
             {

src/SolutionInfo.cs

@@ -2,7 +2,7 @@
 using System.Resources;
 
 [assembly: AssemblyCompany("Umbraco")]
-[assembly: AssemblyCopyright("Copyright © Umbraco 2016")]
+[assembly: AssemblyCopyright("Copyright © Umbraco 2017")]
 [assembly: AssemblyTrademark("")]
 [assembly: AssemblyCulture("")]
 
@@ -11,5 +11,5 @@ using System.Resources;
 
 [assembly: AssemblyVersion("1.0.*")]
 
-[assembly: AssemblyFileVersion("7.5.4")]
-[assembly: AssemblyInformationalVersion("7.5.4")]
+[assembly: AssemblyFileVersion("7.5.7")]
+[assembly: AssemblyInformationalVersion("7.5.7")]

src/Umbraco.Core/Configuration/UmbracoSettings/ContentElement.cs

@@ -4,8 +4,7 @@ using System.Configuration;
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-
-    internal class ContentElement : ConfigurationElement, IContentSection
+    internal class ContentElement : UmbracoConfigurationElement, IContentSection
     {
         [ConfigurationProperty("imaging")]
         internal ContentImagingElement Imaging
@@ -22,25 +21,13 @@ namespace Umbraco.Core.Configuration.UmbracoSettings
         [ConfigurationProperty("ResolveUrlsFromTextString")]
         internal InnerTextConfigurationElement<bool> ResolveUrlsFromTextString
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                       (InnerTextConfigurationElement<bool>)this["ResolveUrlsFromTextString"],
-                        //set the default
-                       false);
-            }
+            get { return GetOptionalTextElement<bool>("ResolveUrlsFromTextString", false); }
         }
 
         [ConfigurationProperty("UploadAllowDirectories")]
         internal InnerTextConfigurationElement<bool> UploadAllowDirectories
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                       (InnerTextConfigurationElement<bool>)this["UploadAllowDirectories"],
-                    //set the default
-                       true);
-            }
+            get { return GetOptionalTextElement("UploadAllowDirectories", true); }
         }
 
         public IEnumerable<IContentErrorPage> Error404Collection
@@ -63,121 +50,61 @@ namespace Umbraco.Core.Configuration.UmbracoSettings
         [ConfigurationProperty("ensureUniqueNaming")]
         internal InnerTextConfigurationElement<bool> EnsureUniqueNaming
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                      (InnerTextConfigurationElement<bool>)this["ensureUniqueNaming"],
-                    //set the default
-                      true);
-            }
+            get { return GetOptionalTextElement("ensureUniqueNaming", true); }
         }
 
         [ConfigurationProperty("TidyEditorContent")]
         internal InnerTextConfigurationElement<bool> TidyEditorContent
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                      (InnerTextConfigurationElement<bool>)this["TidyEditorContent"],
-                    //set the default
-                      false);
-            }
+            get { return GetOptionalTextElement("TidyEditorContent", false); }
         }
 
         [ConfigurationProperty("TidyCharEncoding")]
         internal InnerTextConfigurationElement<string> TidyCharEncoding
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                          (InnerTextConfigurationElement<string>)this["TidyCharEncoding"],
-                            //set the default
-                          "UTF8");
-            }
+            get { return GetOptionalTextElement("TidyCharEncoding", "UTF8"); }
         }
 
         [ConfigurationProperty("XmlCacheEnabled")]
         internal InnerTextConfigurationElement<bool> XmlCacheEnabled
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["XmlCacheEnabled"],
-                    //set the default
-                          true);
-            }
+            get { return GetOptionalTextElement("XmlCacheEnabled", true); }
         }
 
         [ConfigurationProperty("ContinouslyUpdateXmlDiskCache")]
         internal InnerTextConfigurationElement<bool> ContinouslyUpdateXmlDiskCache
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["ContinouslyUpdateXmlDiskCache"],
-                    //set the default
-                          true);                
-            }
+            get { return GetOptionalTextElement("ContinouslyUpdateXmlDiskCache", true); }
         }
 
         [ConfigurationProperty("XmlContentCheckForDiskChanges")]
         internal InnerTextConfigurationElement<bool> XmlContentCheckForDiskChanges
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["XmlContentCheckForDiskChanges"],
-                    //set the default
-                          false); 
-            }
+            get { return GetOptionalTextElement("XmlContentCheckForDiskChanges", false); }
         }
 
         [ConfigurationProperty("EnableSplashWhileLoading")]
         internal InnerTextConfigurationElement<bool> EnableSplashWhileLoading
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["EnableSplashWhileLoading"],
-                    //set the default
-                          false); 
-            }
+            get { return GetOptionalTextElement("EnableSplashWhileLoading", false); }
         }
 
         [ConfigurationProperty("PropertyContextHelpOption")]
         internal InnerTextConfigurationElement<string> PropertyContextHelpOption
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                          (InnerTextConfigurationElement<string>)this["PropertyContextHelpOption"],
-                    //set the default
-                          "text"); 
-            }
+            get { return GetOptionalTextElement("PropertyContextHelpOption", "text"); }
         }
 
         [ConfigurationProperty("UseLegacyXmlSchema")]
         internal InnerTextConfigurationElement<bool> UseLegacyXmlSchema
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["UseLegacyXmlSchema"],
-                    //set the default
-                          false); 
-            }
+            get { return GetOptionalTextElement("UseLegacyXmlSchema", false); }
         }
         
         [ConfigurationProperty("ForceSafeAliases")]
         internal InnerTextConfigurationElement<bool> ForceSafeAliases
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                          (InnerTextConfigurationElement<bool>)this["ForceSafeAliases"],
-                    //set the default
-                          true); 
-            }
+            get { return GetOptionalTextElement("ForceSafeAliases", true); }
         }
 
         [ConfigurationProperty("PreviewBadge")]
@@ -185,123 +112,63 @@ namespace Umbraco.Core.Configuration.UmbracoSettings
         {
             get
             {
-                return new OptionalInnerTextConfigurationElement<string>(
-                          (InnerTextConfigurationElement<string>)this["PreviewBadge"],
-                    //set the default
-                          @"<a id=""umbracoPreviewBadge"" style=""position: absolute; top: 0; right: 0; border: 0; width: 149px; height: 149px; background: url('{1}/preview/previewModeBadge.png') no-repeat;"" href=""{0}/endPreview.aspx?redir={2}""><span style=""display:none;"">In Preview Mode - click to end</span></a>"); 
+                return GetOptionalTextElement("PreviewBadge", @"<a id=""umbracoPreviewBadge"" style=""position: absolute; top: 0; right: 0; border: 0; width: 149px; height: 149px; background: url('{1}/preview/previewModeBadge.png') no-repeat;"" href=""{0}/endPreview.aspx?redir={2}""><span style=""display:none;"">In Preview Mode - click to end</span></a>");                
             }
         }
 
         [ConfigurationProperty("UmbracoLibraryCacheDuration")]
         internal InnerTextConfigurationElement<int> UmbracoLibraryCacheDuration
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<int>(
-                          (InnerTextConfigurationElement<int>)this["UmbracoLibraryCacheDuration"],
-                    //set the default
-                          1800); 
-                
-            }
+            get { return GetOptionalTextElement("UmbracoLibraryCacheDuration", 1800); }
         }
 
         [ConfigurationProperty("MacroErrors")]
         internal InnerTextConfigurationElement<MacroErrorBehaviour> MacroErrors
         {
-            get
-            {
-
-                return new OptionalInnerTextConfigurationElement<MacroErrorBehaviour>(
-                          (InnerTextConfigurationElement<MacroErrorBehaviour>)this["MacroErrors"],
-                    //set the default
-                          MacroErrorBehaviour.Inline); 
-            }
+            get { return GetOptionalTextElement("MacroErrors", MacroErrorBehaviour.Inline); }
         }
 
         [Obsolete("This is here so that if this config element exists we won't get a YSOD, it is not used whatsoever and will be removed in future versions")]
         [ConfigurationProperty("DocumentTypeIconList")]
         internal InnerTextConfigurationElement<IconPickerBehaviour> DocumentTypeIconList
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<IconPickerBehaviour>(
-                          (InnerTextConfigurationElement<IconPickerBehaviour>)this["DocumentTypeIconList"],
-                    //set the default
-                          IconPickerBehaviour.HideFileDuplicates);
-            }
+            get { return GetOptionalTextElement("DocumentTypeIconList", IconPickerBehaviour.HideFileDuplicates); }
         }
 
         [ConfigurationProperty("disallowedUploadFiles")]
         internal CommaDelimitedConfigurationElement DisallowedUploadFiles
         {
-            get
-            {
-                return new OptionalCommaDelimitedConfigurationElement(
-                       (CommaDelimitedConfigurationElement)this["disallowedUploadFiles"],
-                    //set the default
-                       new[] { "ashx", "aspx", "ascx", "config", "cshtml", "vbhtml", "asmx", "air", "axd" });
-
-            }
+            get { return GetOptionalDelimitedElement("disallowedUploadFiles", new[] {"ashx", "aspx", "ascx", "config", "cshtml", "vbhtml", "asmx", "air", "axd"}); }
         }
 
         [ConfigurationProperty("cloneXmlContent")]
         internal InnerTextConfigurationElement<bool> CloneXmlContent
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["cloneXmlContent"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("cloneXmlContent", true); }
         }
 
         [ConfigurationProperty("GlobalPreviewStorageEnabled")]
         internal InnerTextConfigurationElement<bool> GlobalPreviewStorageEnabled
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["GlobalPreviewStorageEnabled"],
-                    //set the default
-                    false);
-            }
+            get { return GetOptionalTextElement("GlobalPreviewStorageEnabled", false); }
         }
 
         [ConfigurationProperty("defaultDocumentTypeProperty")]
         internal InnerTextConfigurationElement<string> DefaultDocumentTypeProperty
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                    (InnerTextConfigurationElement<string>)this["defaultDocumentTypeProperty"],
-                    //set the default
-                    "Textstring");
-            }
+            get { return GetOptionalTextElement("defaultDocumentTypeProperty", "Textstring"); }
         }
 
         [ConfigurationProperty("EnableInheritedDocumentTypes")]
         internal InnerTextConfigurationElement<bool> EnableInheritedDocumentTypes
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>) this["EnableInheritedDocumentTypes"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("EnableInheritedDocumentTypes", true); }
         }
 
         [ConfigurationProperty("EnableInheritedMediaTypes")]
         internal InnerTextConfigurationElement<bool> EnableInheritedMediaTypes
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["EnableInheritedMediaTypes"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("EnableInheritedMediaTypes", true); }
         }
 
         string IContentSection.NotificationEmailAddress

src/Umbraco.Core/Configuration/UmbracoSettings/ContentScriptEditorElement.cs

@@ -3,42 +3,24 @@ using System.Configuration;
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class ContentScriptEditorElement : ConfigurationElement
+    internal class ContentScriptEditorElement : UmbracoConfigurationElement
     {
         [ConfigurationProperty("scriptFolderPath")]
         internal InnerTextConfigurationElement<string> ScriptFolderPath
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                       (InnerTextConfigurationElement<string>)this["scriptFolderPath"],
-                    //set the default
-                       "/scripts");
-            }
+            get { return GetOptionalTextElement("scriptFolderPath", "/scripts"); }
         }
 
         [ConfigurationProperty("scriptFileTypes")]
         internal OptionalCommaDelimitedConfigurationElement ScriptFileTypes
         {
-            get
-            {
-                return new OptionalCommaDelimitedConfigurationElement(
-                       (OptionalCommaDelimitedConfigurationElement)this["scriptFileTypes"],
-                    //set the default
-                       new[] { "js", "xml" });                
-            }
+            get { return GetOptionalDelimitedElement("scriptFileTypes", new[] {"js", "xml"}); }
         }
 
         [ConfigurationProperty("scriptDisableEditor")]
         internal InnerTextConfigurationElement<bool> ScriptEditorDisable
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>) this["scriptDisableEditor"],
-                    //set the default
-                    false);
-            }
+            get { return GetOptionalTextElement("scriptDisableEditor", false); }
         }
 
     }

src/Umbraco.Core/Configuration/UmbracoSettings/ImagingAutoFillUploadFieldElement.cs

@@ -2,7 +2,7 @@
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class ImagingAutoFillUploadFieldElement : ConfigurationElement, IImagingAutoFillUploadField
+    internal class ImagingAutoFillUploadFieldElement : UmbracoConfigurationElement, IImagingAutoFillUploadField
     {
         /// <summary>
         /// Allow setting internally so we can create a default
@@ -17,49 +17,25 @@ namespace Umbraco.Core.Configuration.UmbracoSettings
         [ConfigurationProperty("widthFieldAlias")]
         internal InnerTextConfigurationElement<string> WidthFieldAlias
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                       (InnerTextConfigurationElement<string>)this["widthFieldAlias"],
-                    //set the default
-                       "umbracoWidth");
-            }
+            get { return GetOptionalTextElement("widthFieldAlias", "umbracoWidth"); }
         }
 
         [ConfigurationProperty("heightFieldAlias")]
         internal InnerTextConfigurationElement<string> HeightFieldAlias
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                       (InnerTextConfigurationElement<string>)this["heightFieldAlias"],
-                    //set the default
-                       "umbracoHeight");
-            }
+            get { return GetOptionalTextElement("heightFieldAlias", "umbracoHeight"); }
         }
 
         [ConfigurationProperty("lengthFieldAlias")]
         internal InnerTextConfigurationElement<string> LengthFieldAlias
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                       (InnerTextConfigurationElement<string>)this["lengthFieldAlias"],
-                    //set the default
-                       "umbracoBytes");
-            }
+            get { return GetOptionalTextElement("lengthFieldAlias", "umbracoBytes"); }
         }
 
         [ConfigurationProperty("extensionFieldAlias")]
         internal InnerTextConfigurationElement<string> ExtensionFieldAlias
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                       (InnerTextConfigurationElement<string>)this["extensionFieldAlias"],
-                    //set the default
-                       "umbracoExtension");
-            }
+            get { return GetOptionalTextElement("extensionFieldAlias", "umbracoExtension"); }
         }
 
         string IImagingAutoFillUploadField.Alias

src/Umbraco.Core/Configuration/UmbracoSettings/LoggingElement.cs

@@ -3,67 +3,37 @@ using System.Configuration;
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class LoggingElement : ConfigurationElement, ILoggingSection
+    internal class LoggingElement : UmbracoConfigurationElement, ILoggingSection
     {
         
         [ConfigurationProperty("autoCleanLogs")]
         internal InnerTextConfigurationElement<bool> AutoCleanLogs
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["autoCleanLogs"],
-                    //set the default
-                    false);            
-            }
+            get { return GetOptionalTextElement("autoCleanLogs", false); }
         }
 
         [ConfigurationProperty("enableLogging")]
         internal InnerTextConfigurationElement<bool> EnableLogging
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["enableLogging"],
-                    //set the default
-                    true);            
-            }
+            get { return GetOptionalTextElement("enableLogging", true); }
         }
 
         [ConfigurationProperty("enableAsyncLogging")]
         internal InnerTextConfigurationElement<bool> EnableAsyncLogging
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["enableAsyncLogging"],
-                    //set the default
-                    true);           
-            }
+            get { return GetOptionalTextElement("enableAsyncLogging", true); }
         }
 
         [ConfigurationProperty("cleaningMiliseconds")]
         internal InnerTextConfigurationElement<int> CleaningMiliseconds
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<int>(
-                    (InnerTextConfigurationElement<int>)this["cleaningMiliseconds"],
-                    //set the default
-                    -1);                      
-            }
+            get { return GetOptionalTextElement("cleaningMiliseconds", -1); }
         }
 
         [ConfigurationProperty("maxLogAge")]
         internal InnerTextConfigurationElement<int> MaxLogAge
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<int>(
-                    (InnerTextConfigurationElement<int>)this["maxLogAge"],
-                    //set the default
-                    -1);
-            }
+            get { return GetOptionalTextElement("maxLogAge", -1); }
         }
 
         [ConfigurationCollection(typeof(DisabledLogTypesCollection), AddItemName = "logTypeAlias")]

src/Umbraco.Core/Configuration/UmbracoSettings/NotificationsElement.cs

@@ -2,7 +2,7 @@
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class NotificationsElement : ConfigurationElement
+    internal class NotificationsElement : UmbracoConfigurationElement
     {
         [ConfigurationProperty("email")]
         internal InnerTextConfigurationElement<string> NotificationEmailAddress
@@ -13,13 +13,7 @@ namespace Umbraco.Core.Configuration.UmbracoSettings
         [ConfigurationProperty("disableHtmlEmail")]
         internal InnerTextConfigurationElement<bool> DisableHtmlEmail
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>) this["disableHtmlEmail"], 
-                    //set the default
-                    false);
-            }
+            get { return GetOptionalTextElement("disableHtmlEmail", false); }
         }
 
     }

src/Umbraco.Core/Configuration/UmbracoSettings/RequestHandlerElement.cs

@@ -5,30 +5,18 @@ using System.Collections.Generic;
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class RequestHandlerElement : ConfigurationElement, IRequestHandlerSection
+    internal class RequestHandlerElement : UmbracoConfigurationElement, IRequestHandlerSection
     {
         [ConfigurationProperty("useDomainPrefixes")]
         public InnerTextConfigurationElement<bool> UseDomainPrefixes
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["useDomainPrefixes"],
-                    //set the default
-                    false);  
-            }
+            get { return GetOptionalTextElement("useDomainPrefixes", false); }
         }
 
         [ConfigurationProperty("addTrailingSlash")]
         public InnerTextConfigurationElement<bool> AddTrailingSlash
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["addTrailingSlash"],
-                    //set the default
-                    true);  
-            }
+            get { return GetOptionalTextElement("addTrailingSlash", true); }
         }
 
         private UrlReplacingElement _defaultUrlReplacing;

src/Umbraco.Core/Configuration/UmbracoSettings/SecurityElement.cs

@@ -2,66 +2,36 @@
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class SecurityElement : ConfigurationElement, ISecuritySection
+    internal class SecurityElement : UmbracoConfigurationElement, ISecuritySection
     {
         [ConfigurationProperty("keepUserLoggedIn")]
         internal InnerTextConfigurationElement<bool> KeepUserLoggedIn
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["keepUserLoggedIn"],
-                    //set the default
-                    true);   
-            }
+            get { return GetOptionalTextElement("keepUserLoggedIn", true); }
         }
 
         [ConfigurationProperty("hideDisabledUsersInBackoffice")]
         internal InnerTextConfigurationElement<bool> HideDisabledUsersInBackoffice
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["hideDisabledUsersInBackoffice"],
-                    //set the default
-                    false);                          
-            }
+            get { return GetOptionalTextElement("hideDisabledUsersInBackoffice", false); }
         }
 
         [ConfigurationProperty("allowPasswordReset")]
         internal InnerTextConfigurationElement<bool> AllowPasswordReset
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["allowPasswordReset"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("allowPasswordReset", true); }
         }
 
         [ConfigurationProperty("authCookieName")]
         internal InnerTextConfigurationElement<string> AuthCookieName
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                    (InnerTextConfigurationElement<string>)this["authCookieName"],
-                    //set the default
-                    Constants.Web.AuthCookieName);                
-            }
+            get { return GetOptionalTextElement("authCookieName", Constants.Web.AuthCookieName); }
         }
 
         [ConfigurationProperty("authCookieDomain")]
         internal InnerTextConfigurationElement<string> AuthCookieDomain
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<string>(
-                    (InnerTextConfigurationElement<string>)this["authCookieDomain"],
-                    //set the default
-                    null);                    
-            }
+            get { return GetOptionalTextElement<string>("authCookieDomain", null); }
         }
 
         bool ISecuritySection.KeepUserLoggedIn

src/Umbraco.Core/Configuration/UmbracoSettings/TemplatesElement.cs

@@ -3,55 +3,31 @@ using System.Configuration;
 
 namespace Umbraco.Core.Configuration.UmbracoSettings
 {
-    internal class TemplatesElement : ConfigurationElement, ITemplatesSection
+    internal class TemplatesElement : UmbracoConfigurationElement, ITemplatesSection
     {
         [ConfigurationProperty("useAspNetMasterPages")]
         internal InnerTextConfigurationElement<bool> UseAspNetMasterPages
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["useAspNetMasterPages"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("useAspNetMasterPages", true); }
         }
 
         [ConfigurationProperty("enableSkinSupport")]
         internal InnerTextConfigurationElement<bool> EnableSkinSupport
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["enableSkinSupport"],
-                    //set the default
-                    true);
-            }
+            get { return GetOptionalTextElement("enableSkinSupport", true); }
         }
 
         [ConfigurationProperty("defaultRenderingEngine", IsRequired = true)]
         internal InnerTextConfigurationElement<RenderingEngine> DefaultRenderingEngine
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<RenderingEngine>(
-                    (InnerTextConfigurationElement<RenderingEngine>)this["defaultRenderingEngine"],
-                    //set the default
-                    RenderingEngine.Mvc);
-            }
+            get { return GetOptionalTextElement("defaultRenderingEngine", RenderingEngine.Mvc); }
         }
 
         [Obsolete("This has no affect and will be removed in future versions")]
         [ConfigurationProperty("enableTemplateFolders")]
         internal InnerTextConfigurationElement<bool> EnableTemplateFolders
         {
-            get
-            {
-                return new OptionalInnerTextConfigurationElement<bool>(
-                    (InnerTextConfigurationElement<bool>)this["enableTemplateFolders"],
-                    //set the default
-                    false);
-            }
+            get { return GetOptionalTextElement("enableTemplateFolders", false); }
         }
 
         bool ITemplatesSection.UseAspNetMasterPages

src/Umbraco.Core/Configuration/UmbracoSettings/UmbracoConfigurationElement.cs

@@ -0,0 +1,36 @@
+using System.Collections.Concurrent;
+using System.Configuration;
+
+namespace Umbraco.Core.Configuration.UmbracoSettings
+{
+    /// <summary>
+    /// Base class with shared helper methods
+    /// </summary>
+    internal class UmbracoConfigurationElement : ConfigurationElement
+    {
+        /// <summary>
+        /// Used so the RawElement types are not re-created every time they are accessed
+        /// </summary>
+        private readonly ConcurrentDictionary<string, RawXmlConfigurationElement> _rawElements = new ConcurrentDictionary<string, RawXmlConfigurationElement>();
+
+        protected OptionalInnerTextConfigurationElement<T> GetOptionalTextElement<T>(string name, T defaultVal)
+        {
+            return (OptionalInnerTextConfigurationElement<T>) _rawElements.GetOrAdd(
+                name,
+                s => new OptionalInnerTextConfigurationElement<T>(
+                    (InnerTextConfigurationElement<T>) this[s],
+                    //set the default
+                    defaultVal));
+        }
+
+        protected OptionalCommaDelimitedConfigurationElement GetOptionalDelimitedElement(string name, string[] defaultVal)
+        {
+            return (OptionalCommaDelimitedConfigurationElement) _rawElements.GetOrAdd(
+                name,
+                s => new OptionalCommaDelimitedConfigurationElement(
+                    (CommaDelimitedConfigurationElement) this[name],
+                    //set the default
+                    defaultVal));
+        }
+    }
+}

src/Umbraco.Core/Configuration/UmbracoVersion.cs

@@ -6,7 +6,7 @@ namespace Umbraco.Core.Configuration
 {
     public class UmbracoVersion
     {
-        private static readonly Version Version = new Version("7.5.4");
+        private static readonly Version Version = new Version("7.5.7");
 
         /// <summary>
         /// Gets the current version of Umbraco.

src/Umbraco.Core/Constants-Conventions.cs

@@ -122,6 +122,11 @@ namespace Umbraco.Core
 				/// MediaType alias for an image.
 				/// </summary>
 				public const string Image = "Image";
+
+                /// <summary>
+                /// MediaType alias indicating allowing auto-selection.
+                /// </summary>
+			    public const string AutoSelect = "umbracoAutoSelect";
 			}
             
 		    /// <summary>

src/Umbraco.Core/CoreBootManager.cs

@@ -1,5 +1,6 @@
 using System;
 using System.Collections.Generic;
+using System.Configuration;
 using System.IO;
 using System.Linq;
 using System.Threading;
@@ -30,12 +31,13 @@ using Umbraco.Core.Manifest;
 using Umbraco.Core.Services;
 using Umbraco.Core.Sync;
 using Umbraco.Core.Strings;
+using IntegerValidator = Umbraco.Core.PropertyEditors.IntegerValidator;
 using MigrationsVersionFourNineZero = Umbraco.Core.Persistence.Migrations.Upgrades.TargetVersionFourNineZero;
 
 namespace Umbraco.Core
 {
     /// <summary>
-    /// A bootstrapper for the Umbraco application which initializes all objects for the Core of the application 
+    /// A bootstrapper for the Umbraco application which initializes all objects for the Core of the application
     /// </summary>
     /// <remarks>
     /// This does not provide any startup functionality relating to web objects
@@ -191,14 +193,14 @@ namespace Umbraco.Core
         protected virtual CacheHelper CreateApplicationCache()
         {
             var cacheHelper = new CacheHelper(
-                //we need to have the dep clone runtime cache provider to ensure 
+                //we need to have the dep clone runtime cache provider to ensure
                 //all entities are cached properly (cloned in and cloned out)
                 new DeepCloneRuntimeCacheProvider(new ObjectCacheRuntimeCacheProvider()),
                 new StaticCacheProvider(),
                 //we have no request based cache when not running in web-based context
                 new NullCacheProvider(),
                 new IsolatedRuntimeCache(type =>
-                    //we need to have the dep clone runtime cache provider to ensure 
+                    //we need to have the dep clone runtime cache provider to ensure
                     //all entities are cached properly (cloned in and cloned out)
                     new DeepCloneRuntimeCacheProvider(new ObjectCacheRuntimeCacheProvider())));
 
@@ -251,18 +253,18 @@ namespace Umbraco.Core
         }
 
         /// <summary>
-        /// Special method to initialize the ApplicationEventsResolver and any modifications required for it such 
+        /// Special method to initialize the ApplicationEventsResolver and any modifications required for it such
         /// as adding custom types to the resolver.
         /// </summary>
         protected virtual void InitializeApplicationEventsResolver()
         {
             //find and initialize the application startup handlers, we need to initialize this resolver here because
-            //it is a special resolver where they need to be instantiated first before any other resolvers in order to bind to 
+            //it is a special resolver where they need to be instantiated first before any other resolvers in order to bind to
             //events and to call their events during bootup.
             //ApplicationStartupHandler.RegisterHandlers();
             //... and set the special flag to let us resolve before frozen resolution
             ApplicationEventsResolver.Current = new ApplicationEventsResolver(
-                ServiceProvider, 
+                ServiceProvider,
                 ProfilingLogger.Logger,
                 PluginManager.ResolveApplicationStartupHandlers())
             {
@@ -282,7 +284,7 @@ namespace Umbraco.Core
         }
 
         /// <summary>
-        /// Fires after initialization and calls the callback to allow for customizations to occur & 
+        /// Fires after initialization and calls the callback to allow for customizations to occur &
         /// Ensure that the OnApplicationStarting methods of the IApplicationEvents are called
         /// </summary>
         /// <param name="afterStartup"></param>
@@ -334,7 +336,7 @@ namespace Umbraco.Core
         {
             if (_isComplete)
                 throw new InvalidOperationException("The boot manager has already been completed");
-            
+
             FreezeResolution();
 
             //Here we need to make sure the db can be connected to
@@ -366,7 +368,7 @@ namespace Umbraco.Core
                             ProfilingLogger.Logger.Error<CoreBootManager>("An error occurred running OnApplicationStarted for handler " + x.GetType(), ex);
                             throw;
                         }
-                    }); 
+                    });
             }
 
             //Now, startup all of our legacy startup handler
@@ -455,6 +457,10 @@ namespace Umbraco.Core
             {
                 ServerRegistrarResolver.Current = new ServerRegistrarResolver(new ConfigServerRegistrar());
             }
+            else if ("true".InvariantEquals(ConfigurationManager.AppSettings["umbracoDisableElectionForSingleServer"]))
+            {
+                ServerRegistrarResolver.Current = new ServerRegistrarResolver(new SingleServerRegistrar());
+            }
             else
             {
                 ServerRegistrarResolver.Current = new ServerRegistrarResolver(
@@ -462,7 +468,6 @@ namespace Umbraco.Core
                         new Lazy<IServerRegistrationService>(() => ApplicationContext.Services.ServerRegistrationService),
                         new DatabaseServerRegistrarOptions()));
             }
-            
 
             //by default we'll use the database server messenger with default options (no callbacks),
             // this will be overridden in the web startup
@@ -473,7 +478,7 @@ namespace Umbraco.Core
                 ServiceProvider, ProfilingLogger.Logger,
                 () => PluginManager.ResolveAssignedMapperTypes());
 
-           
+
             //RepositoryResolver.Current = new RepositoryResolver(
             //    new RepositoryFactory(ApplicationCache));
 

src/Umbraco.Core/DatabaseContext.cs

@@ -34,6 +34,12 @@ namespace Umbraco.Core
         private string _connectionString;
         private string _providerName;
         private DatabaseSchemaResult _result;
+        private DateTime? _connectionLastChecked = null;
+
+        /// <summary>
+        /// The number of minutes to throttle the checks to CanConnect
+        /// </summary>
+        private const int ConnectionCheckMinutes = 1;
 
         [Obsolete("Use the constructor specifying all dependencies instead")]
         public DatabaseContext(IDatabaseFactory factory)
@@ -110,13 +116,28 @@ namespace Umbraco.Core
         {
             get
             {
-                if (IsDatabaseConfigured == false) return false;
-                var canConnect = DbConnectionExtensions.IsConnectionAvailable(ConnectionString, DatabaseProvider);
-                LogHelper.Info<DatabaseContext>("CanConnect = " + canConnect);
-                return canConnect;
+                if (IsDatabaseConfigured == false)
+                    return false;
+
+                //Don't check again if the timeout period hasn't elapsed
+                //this ensures we don't keep checking the connection too many times in a row like during startup.
+                //Do check if the _connectionLastChecked is null which means we're just initializing or it could 
+                //not connect last time it was checked.
+                if ((_connectionLastChecked.HasValue && (DateTime.Now - _connectionLastChecked.Value).TotalMinutes > ConnectionCheckMinutes)
+                    || _connectionLastChecked.HasValue == false)
+                {
+                    var canConnect = DbConnectionExtensions.IsConnectionAvailable(ConnectionString, DatabaseProvider);
+                    LogHelper.Info<DatabaseContext>("CanConnect = " + canConnect);
+
+                    _connectionLastChecked = canConnect == false ? null : (DateTime?) DateTime.Now;
+                    return canConnect;
+                }
+
+                return _connectionLastChecked.HasValue;
             }
         }
 
+
         /// <summary>
         /// Gets the configured umbraco db connection string.
         /// </summary>
@@ -174,7 +195,7 @@ namespace Umbraco.Core
         /// </summary>
         public void ConfigureEmbeddedDatabaseConnection()
         {
-            const string providerName = "System.Data.SqlServerCe.4.0";
+            const string providerName = Constants.DatabaseProviders.SqlCe;
 
             var connectionString = GetEmbeddedDatabaseConnectionString();
             SaveConnectionString(connectionString, providerName);

src/Umbraco.Core/Logging/ImageProcessorLogger.cs

@@ -1,9 +1,9 @@
-namespace Umbraco.Core.Logging
-{
-    using System;
-    using System.Runtime.CompilerServices;
+using System;
+using System.Runtime.CompilerServices;
+using ImageProcessor.Common.Exceptions;
 
-    using ImageProcessor.Common.Exceptions;
+namespace Umbraco.Core.Logging
+{   
 
     /// <summary>
     /// A logger for explicitly logging ImageProcessor exceptions.

src/Umbraco.Core/MainDom.cs

@@ -1,18 +1,21 @@
 using System;
 using System.Collections.Generic;
-using System.Diagnostics;
-using System.IO.MemoryMappedFiles;
-using System.Text;
 using System.Threading;
-using System.Threading.Tasks;
 using System.Web.Hosting;
 using Umbraco.Core.Logging;
-using Umbraco.Core.ObjectResolution;
 
 namespace Umbraco.Core
 {
-    // represents the main domain
-    class MainDom : IRegisteredObject
+    /// <summary>
+    /// Represents the main AppDomain running for a given application.
+    /// </summary>
+    /// <remarks>
+    /// <para>There can be only one "main" AppDomain running for a given application at a time.</para>
+    /// <para>When an AppDomain starts, it tries to acquire the main domain status.</para>
+    /// <para>When an AppDomain stops (eg the application is restarting) it should release the main domain status.</para>
+    /// <para>It is possible to register against the MainDom and be notified when it is released.</para>
+    /// </remarks>
+    internal class MainDom : IRegisteredObject
     {
         #region Vars
 
@@ -25,7 +28,7 @@ namespace Umbraco.Core
         private readonly AsyncLock _asyncLock;
         private IDisposable _asyncLocker;
 
-        // event wait handle used to notify current main domain that it should 
+        // event wait handle used to notify current main domain that it should
         // release the lock because a new domain wants to be the main domain
         private readonly EventWaitHandle _signal;
 
@@ -34,16 +37,26 @@ namespace Umbraco.Core
         private volatile bool _signaled; // we have been signaled
 
         // actions to run before releasing the main domain
-        private readonly SortedList<int, Action> _callbacks = new SortedList<int, Action>();
+        private readonly SortedList<int, Action> _callbacks = new SortedList<int, Action>(new WeightComparer());
 
         private const int LockTimeoutMilliseconds = 90000; // (1.5 * 60 * 1000) == 1 min 30 seconds
 
+        private class WeightComparer : IComparer<int>
+        {
+            public int Compare(int x, int y)
+            {
+                var result = x.CompareTo(y);
+                // return "equal" as "greater than"
+                return result == 0 ? 1 : result;
+            }
+        }
+
         #endregion
 
         #region Ctor
 
         // initializes a new instance of MainDom
-        public MainDom(ILogger logger)
+        internal MainDom(ILogger logger)
         {
             _logger = logger;
 
@@ -52,22 +65,47 @@ namespace Umbraco.Core
             if (HostingEnvironment.ApplicationID != null)
                 appId = HostingEnvironment.ApplicationID.ReplaceNonAlphanumericChars(string.Empty);
 
-            var lockName = "UMBRACO-" + appId + "-MAINDOM-LCK";
+            // combining with the physical path because if running on eg IIS Express,
+            // two sites could have the same appId even though they are different.
+            //
+            // now what could still collide is... two sites, running in two different processes
+            // and having the same appId, and running on the same app physical path
+            //
+            // we *cannot* use the process ID here because when an AppPool restarts it is
+            // a new process for the same application path
+
+            var appPath = HostingEnvironment.ApplicationPhysicalPath;
+            var hash = (appId + ":::" + appPath).ToSHA1();
+
+            var lockName = "UMBRACO-" + hash + "-MAINDOM-LCK";
             _asyncLock = new AsyncLock(lockName);
 
-            var eventName = "UMBRACO-" + appId + "-MAINDOM-EVT";
+            var eventName = "UMBRACO-" + hash + "-MAINDOM-EVT";
             _signal = new EventWaitHandle(false, EventResetMode.AutoReset, eventName);
         }
 
         #endregion
 
-        // register a main domain consumer
+        /// <summary>
+        /// Registers a resource that requires the current AppDomain to be the main domain to function.
+        /// </summary>
+        /// <param name="release">An action to execute before the AppDomain releases the main domain status.</param>
+        /// <param name="weight">An optional weight (lower goes first).</param>
+        /// <returns>A value indicating whether it was possible to register.</returns>
         public bool Register(Action release, int weight = 100)
         {
             return Register(null, release, weight);
         }
 
-        // register a main domain consumer
+        /// <summary>
+        /// Registers a resource that requires the current AppDomain to be the main domain to function.
+        /// </summary>
+        /// <param name="install">An action to execute when registering.</param>
+        /// <param name="release">An action to execute before the AppDomain releases the main domain status.</param>
+        /// <param name="weight">An optional weight (lower goes first).</param>
+        /// <returns>A value indicating whether it was possible to register.</returns>
+        /// <remarks>If registering is successful, then the <paramref name="install"/> action
+        /// is guaranteed to execute before the AppDomain releases the main domain status.</remarks>
         public bool Register(Action install, Action release, int weight = 100)
         {
             lock (_locko)
@@ -97,7 +135,7 @@ namespace Umbraco.Core
 
             try
             {
-                _logger.Debug<MainDom>("Stopping...");
+                _logger.Info<MainDom>("Stopping...");
                 foreach (var callback in _callbacks.Values)
                 {
                     try
@@ -109,7 +147,7 @@ namespace Umbraco.Core
                         _logger.Error<MainDom>("Error while running callback, remaining callbacks will not run.", e);
                         throw;
                     }
-                    
+
                 }
                 _logger.Debug<MainDom>("Stopped.");
             }
@@ -118,12 +156,12 @@ namespace Umbraco.Core
                 // in any case...
                 _isMainDom = false;
                 _asyncLocker.Dispose();
-                _logger.Debug<MainDom>("Released MainDom.");
+                _logger.Info<MainDom>("Released MainDom.");
             }
         }
 
         // acquires the main domain
-        public bool Acquire()
+        internal bool Acquire()
         {
             lock (_locko) // we don't want the hosting environment to interfere by signaling
             {
@@ -131,11 +169,11 @@ namespace Umbraco.Core
                 // the handler is not installed so that would be the hosting environment
                 if (_signaled)
                 {
-                    _logger.Debug<MainDom>("Cannot acquire MainDom (signaled).");
+                    _logger.Info<MainDom>("Cannot acquire MainDom (signaled).");
                     return false;
                 }
 
-                _logger.Debug<MainDom>("Acquiring MainDom...");
+                _logger.Info<MainDom>("Acquiring MainDom...");
 
                 // signal other instances that we want the lock, then wait one the lock,
                 // which may timeout, and this is accepted - see comments below
@@ -162,7 +200,7 @@ namespace Umbraco.Core
 
                 HostingEnvironment.RegisterObject(this);
 
-                _logger.Debug<MainDom>("Acquired MainDom.");
+                _logger.Info<MainDom>("Acquired MainDom.");
                 return true;
             }
         }
@@ -174,7 +212,7 @@ namespace Umbraco.Core
         }
 
         // IRegisteredObject
-        public void Stop(bool immediate)
+        void IRegisteredObject.Stop(bool immediate)
         {
             try
             {

src/Umbraco.Core/Models/ContentExtensions.cs

@@ -404,8 +404,7 @@ namespace Umbraco.Core.Models
             return content.Properties
                           .Where(property => propertyGroup.PropertyTypes
                                                           .Select(propertyType => propertyType.Id)
-                                                          .Contains(property.PropertyTypeId))
-                          .OrderBy(x => x.PropertyType.SortOrder);
+                                                          .Contains(property.PropertyTypeId));
         }
 
         /// <summary>

src/Umbraco.Core/Models/ContentTypeBase.cs

@@ -326,7 +326,7 @@ namespace Umbraco.Core.Models
             }
         }
 
-            /// <summary>
+        /// <summary>
         /// A boolean flag indicating if a property type has been removed from this instance.
         /// </summary>
         /// <remarks>

src/Umbraco.Core/Models/PublicAccessEntry.cs

@@ -25,7 +25,7 @@ namespace Umbraco.Core.Models
             NoAccessNodeId = noAccessNode.Id;
             _protectedNodeId = protectedNode.Id;
 
-            _ruleCollection = new ObservableCollection<PublicAccessRule>(ruleCollection);            
+            _ruleCollection = new ObservableCollection<PublicAccessRule>(ruleCollection);
             _ruleCollection.CollectionChanged += _ruleCollection_CollectionChanged;
         }
 
@@ -81,7 +81,7 @@ namespace Umbraco.Core.Models
         internal IEnumerable<Guid> RemovedRules
         {
             get { return _removedRules; }
-        } 
+        }
 
         public IEnumerable<PublicAccessRule> Rules
         {
@@ -107,10 +107,7 @@ namespace Umbraco.Core.Models
 
         public void ClearRules()
         {
-            for (var i = _ruleCollection.Count - 1; i >= 0; i--)
-            {
-                RemoveRule(_ruleCollection[i]);
-            }
+            _ruleCollection.Clear();
         }
 
         [DataMember]
@@ -126,7 +123,7 @@ namespace Umbraco.Core.Models
             get { return _noAccessNodeId; }
             set { SetPropertyValueAndDetectChanges(value, ref _noAccessNodeId, Ps.Value.NoAccessNodeIdSelector); }
         }
-       
+
         [DataMember]
         public int ProtectedNodeId
         {

src/Umbraco.Core/Models/Rdbms/PropertyDataDto.cs

@@ -11,7 +11,6 @@ namespace Umbraco.Core.Models.Rdbms
     {
         [Column("id")]
         [PrimaryKeyColumn]
-        [Index(IndexTypes.NonClustered, Name = "IX_cmsPropertyData")]
         public int Id { get; set; }
 
         [Column("contentNodeId")]

src/Umbraco.Core/Models/UmbracoEntityExtensions.cs

@@ -1,15 +1,117 @@
 using System;
 using System.Collections.Generic;
+using System.IO;
 using System.Linq;
 using System.Text;
 using System.Threading.Tasks;
 using System.Web.UI.WebControls;
+using Umbraco.Core.Logging;
 using Umbraco.Core.Models.EntityBase;
+using Umbraco.Core.Models.Rdbms;
 
 namespace Umbraco.Core.Models
 {
     internal static class UmbracoEntityExtensions
     {
+        /// <summary>
+        /// Does a quick check on the entity's set path to ensure that it's valid and consistent
+        /// </summary>
+        /// <param name="entity"></param>
+        /// <returns></returns>
+        public static void ValidatePathWithException(this NodeDto entity)
+        {
+            //don't validate if it's empty and it has no id
+            if (entity.NodeId == default(int) && entity.Path.IsNullOrWhiteSpace())
+                return;
+
+            if (entity.Path.IsNullOrWhiteSpace())
+                throw new InvalidDataException(string.Format("The content item {0} has an empty path: {1} with parentID: {2}", entity.NodeId, entity.Path, entity.ParentId));
+
+            var pathParts = entity.Path.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
+            if (pathParts.Length < 2)
+            {
+                //a path cannot be less than 2 parts, at a minimum it must be root (-1) and it's own id
+                throw new InvalidDataException(string.Format("The content item {0} has an invalid path: {1} with parentID: {2}", entity.NodeId, entity.Path, entity.ParentId));
+            }
+
+            if (entity.ParentId != default(int) && pathParts[pathParts.Length - 2] != entity.ParentId.ToInvariantString())
+            {
+                //the 2nd last id in the path must be it's parent id
+                throw new InvalidDataException(string.Format("The content item {0} has an invalid path: {1} with parentID: {2}", entity.NodeId, entity.Path, entity.ParentId));
+            }
+        }
+
+        /// <summary>
+        /// Does a quick check on the entity's set path to ensure that it's valid and consistent
+        /// </summary>
+        /// <param name="entity"></param>
+        /// <returns></returns>
+        public static bool ValidatePath(this IUmbracoEntity entity)
+        {
+            //don't validate if it's empty and it has no id
+            if (entity.HasIdentity == false && entity.Path.IsNullOrWhiteSpace())
+                return true;
+
+            if (entity.Path.IsNullOrWhiteSpace())
+                return false;
+
+            var pathParts = entity.Path.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
+            if (pathParts.Length < 2)
+            {
+                //a path cannot be less than 2 parts, at a minimum it must be root (-1) and it's own id
+                return false;
+            }
+
+            if (entity.ParentId != default(int) && pathParts[pathParts.Length - 2] != entity.ParentId.ToInvariantString())
+            {
+                //the 2nd last id in the path must be it's parent id
+                return false;
+            }
+
+            return true;
+        }
+
+        /// <summary>
+        /// This will validate the entity's path and if it's invalid it will fix it, if fixing is required it will recursively 
+        /// check and fix all ancestors if required.
+        /// </summary>
+        /// <param name="entity"></param>
+        /// <param name="logger"></param>
+        /// <param name="getParent">A callback specified to retrieve the parent entity of the entity</param>
+        /// <param name="update">A callback specified to update a fixed entity</param>
+        public static void EnsureValidPath<T>(this T entity, 
+            ILogger logger, 
+            Func<T, T> getParent,
+            Action<T> update)
+            where T: IUmbracoEntity
+        {
+            if (entity.HasIdentity == false)
+                throw new InvalidOperationException("Could not ensure the entity path, the entity has not been assigned an identity");
+
+            if (entity.ValidatePath() == false)
+            {
+                logger.Warn(typeof(UmbracoEntityExtensions), string.Format("The content item {0} has an invalid path: {1} with parentID: {2}", entity.Id, entity.Path, entity.ParentId));
+                if (entity.ParentId == -1)
+                {
+                    entity.Path = string.Concat("-1,", entity.Id);
+                    //path changed, update it
+                    update(entity);
+                }
+                else
+                {
+                    var parent = getParent(entity);
+                    if (parent == null)
+                        throw new NullReferenceException("Could not ensure path for entity " + entity.Id + " could not resolve it's parent " + entity.ParentId);
+
+                    //the parent must also be valid!
+                    parent.EnsureValidPath(logger, getParent, update);
+
+                    entity.Path = string.Concat(parent.Path, ",", entity.Id);
+                    //path changed, update it
+                    update(entity);
+                }
+            }
+        }
 
         public static bool HasChildren(this IUmbracoEntity entity)
         {

src/Umbraco.Core/ObjectResolution/ApplicationEventsResolver.cs

@@ -7,15 +7,20 @@ using umbraco.interfaces;
 
 namespace Umbraco.Core.ObjectResolution
 {
-	/// <summary>
+    /// <summary>
 	/// A resolver to return all IApplicationEvents objects
 	/// </summary>
 	/// <remarks>
-	/// This is disposable because after the app has started it should be disposed to release any memory being occupied by instances.
+	/// <para>This is disposable because after the app has started it should be disposed to release any memory being occupied by instances.</para>
+	/// <para>Ordering handlers: handlers are ordered by (ascending) weight. By default, handlers from the Umbraco.* or Concorde.*
+	/// assemblies have a -100 weight whereas any other handler has a weight of +100. A custom weight can be assigned to a handler
+	/// by marking the class with the WeightAttribute. For example, the CacheRefresherEventHandler is marked with [Weight(int.MinValue)]
+	/// because its events need to run before anything else. Positive weights are considered "user-space" while negative weights are
+	/// "core". Finally, users can register a filter to process the list (after it has been ordered) and re-order it, or remove handlers.</para>
+	/// <para>BEWARE! handlers order is an important thing, and removing handlers or reordering handlers can have unexpected consequences.</para>
 	/// </remarks>
-    internal  sealed class ApplicationEventsResolver : ManyObjectsResolverBase<ApplicationEventsResolver, IApplicationEventHandler>, IDisposable
+    public sealed class ApplicationEventsResolver : ManyObjectsResolverBase<ApplicationEventsResolver, IApplicationEventHandler>, IDisposable
 	{
-
 	    private readonly LegacyStartupHandlerResolver _legacyResolver;
 
 	    /// <summary>
@@ -23,18 +28,18 @@ namespace Umbraco.Core.ObjectResolution
 	    /// </summary>
 	    /// <param name="logger"></param>
 	    /// <param name="applicationEventHandlers"></param>
-	    /// <param name="serviceProvider"></param>		
+	    /// <param name="serviceProvider"></param>
 	    internal ApplicationEventsResolver(IServiceProvider serviceProvider, ILogger logger, IEnumerable<Type> applicationEventHandlers)
             : base(serviceProvider, logger, applicationEventHandlers)
-		{
+	    {
             //create the legacy resolver and only include the legacy types
-	        _legacyResolver = new LegacyStartupHandlerResolver(
+            _legacyResolver = new LegacyStartupHandlerResolver(
                 serviceProvider, logger,
-	            applicationEventHandlers.Where(x => !TypeHelper.IsTypeAssignableFrom<IApplicationEventHandler>(x)));
+	            applicationEventHandlers.Where(x => TypeHelper.IsTypeAssignableFrom<IApplicationEventHandler>(x) == false));
 		}
 
         /// <summary>
-        /// Override in order to only return types of IApplicationEventHandler and above, 
+        /// Override in order to only return types of IApplicationEventHandler and above,
         /// do not include the legacy types of IApplicationStartupHandler
         /// </summary>
         protected override IEnumerable<Type> InstanceTypes
@@ -42,14 +47,60 @@ namespace Umbraco.Core.ObjectResolution
             get { return base.InstanceTypes.Where(TypeHelper.IsTypeAssignableFrom<IApplicationEventHandler>); }
         }
 
-	    /// <summary>
-		/// Gets the <see cref="IApplicationEventHandler"/> implementations.
-		/// </summary>
-		public IEnumerable<IApplicationEventHandler> ApplicationEventHandlers
+	    private List<IApplicationEventHandler> _orderedAndFiltered;
+
+        /// <summary>
+        /// Gets the <see cref="IApplicationEventHandler"/> implementations.
+        /// </summary>
+        public IEnumerable<IApplicationEventHandler> ApplicationEventHandlers
 		{
-            get { return Values; }
+	        get
+	        {
+	            if (_orderedAndFiltered == null)
+	            {
+                    _orderedAndFiltered = GetSortedValues().ToList();
+                    if (FilterCollection != null)
+                        FilterCollection(_orderedAndFiltered);
+                }
+	            return _orderedAndFiltered;
+	        }
 		}
 
+        /// <summary>
+        /// Gets or sets a delegate to filter the event handler list (EXPERT!).
+        /// </summary>
+        /// <remarks>
+        /// <para>This can be set on startup in the pre-boot process in either a custom boot manager or global.asax (UmbracoApplication).</para>
+        /// <para>Allows custom logic to execute in order to filter and/or re-order the event handlers prior to executing.</para>
+        /// <para>To be used by custom boot sequences where the boot loader needs to remove some handlers, or raise their priority.</para>
+        /// <para>Filtering the event handler collection can have ugly consequences. Use with care.</para>
+        /// </remarks>
+        public Action<IList<IApplicationEventHandler>> FilterCollection
+	    {
+	        get { return _filterCollection; }
+	        set
+	        {
+                if (_orderedAndFiltered != null)
+                    throw new InvalidOperationException("Cannot set the FilterCollection delegate once the ApplicationEventHandlers are resolved");
+                if (_filterCollection != null)
+                    throw new InvalidOperationException("Cannot set the FilterCollection delegate once it's already been specified");
+
+                _filterCollection = value;
+	        }
+	    }
+
+	    protected override int GetObjectWeight(object o)
+	    {
+            var type = o.GetType();
+            var attr = type.GetCustomAttribute<WeightAttribute>(true);
+            if (attr != null) return attr.Weight;
+            var name = type.Assembly.FullName;
+
+            // we should really attribute all our Core handlers, so this is temp
+            var core = name.InvariantStartsWith("Umbraco.") || name.InvariantStartsWith("Concorde.");
+            return core ? -DefaultPluginWeight : DefaultPluginWeight;
+        }
+
         /// <summary>
         /// Create instances of all of the legacy startup handlers
         /// </summary>
@@ -62,11 +113,11 @@ namespace Umbraco.Core.ObjectResolution
 		protected override bool SupportsClear
 		{
             get { return false; }
-		}		
+		}
 
 		protected override bool SupportsInsert
 		{
-			get { return false; }			
+			get { return false; }
 		}
 
 	    private class LegacyStartupHandlerResolver : ManyObjectsResolverBase<ApplicationEventsResolver, IApplicationStartupHandler>, IDisposable
@@ -90,8 +141,9 @@ namespace Umbraco.Core.ObjectResolution
 
 	    private bool _disposed;
 		private readonly ReaderWriterLockSlim _disposalLocker = new ReaderWriterLockSlim();
+	    private Action<IList<IApplicationEventHandler>> _filterCollection;
 
-		/// <summary>
+	    /// <summary>
 		/// Gets a value indicating whether this instance is disposed.
 		/// </summary>
 		/// <value>
@@ -146,7 +198,8 @@ namespace Umbraco.Core.ObjectResolution
 	    {
             _legacyResolver.Dispose();
             ResetCollections();
+            _orderedAndFiltered.Clear();
+	        _orderedAndFiltered = null;
 	    }
-        
-	}
+    }
 }

src/Umbraco.Core/ObjectResolution/LazyManyObjectsResolverbase.cs

@@ -17,7 +17,7 @@ namespace Umbraco.Core.ObjectResolution
     /// for when there is some processing overhead (i.e. Type finding in assemblies) to return the Types used to instantiate the instances. 
     /// In some these cases we don't want to have to type-find during application startup, only when we need to resolve the instances.</para>
     /// <para>Important notes about this resolver: it does not support Insert or Remove and therefore does not support any ordering unless 
-    /// the types are marked with the WeightedPluginAttribute.</para>
+    /// the types are marked with the WeightAttribute.</para>
     /// </remarks>
     public abstract class LazyManyObjectsResolverBase<TResolver, TResolved> : ManyObjectsResolverBase<TResolver, TResolved>
         where TResolved : class

src/Umbraco.Core/ObjectResolution/ManyObjectsResolverBase.cs

@@ -14,18 +14,18 @@ namespace Umbraco.Core.ObjectResolution
 	/// <typeparam name="TResolver">The type of the concrete resolver class.</typeparam>
 	/// <typeparam name="TResolved">The type of the resolved objects.</typeparam>
 	public abstract class ManyObjectsResolverBase<TResolver, TResolved> : ResolverBase<TResolver>
-		where TResolved : class
+        where TResolved : class
         where TResolver : ResolverBase
-	{
-		private Lazy<IEnumerable<TResolved>> _applicationInstances;
-		private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
-		private readonly string _httpContextKey;
-		private readonly List<Type> _instanceTypes = new List<Type>();
-	    private IEnumerable<TResolved> _sortedValues;
+    {
+        private Lazy<IEnumerable<TResolved>> _applicationInstances;
+        private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
+        private readonly string _httpContextKey;
+        private readonly List<Type> _instanceTypes = new List<Type>();
+        private IEnumerable<TResolved> _sortedValues;
 
-		private int _defaultPluginWeight = 10;
+        private int _defaultPluginWeight = 100;
 
-		#region Constructors
+        #region Constructors
 
         /// <summary>
         /// Initializes a new instance of the <see cref="ManyObjectsResolverBase{TResolver, TResolved}"/> class with an empty list of objects,
@@ -61,11 +61,11 @@ namespace Umbraco.Core.ObjectResolution
 
         [EditorBrowsable(EditorBrowsableState.Never)]
         [Obsolete("Use ctor specifying IServiceProvider instead")]
-		protected ManyObjectsResolverBase(ObjectLifetimeScope scope = ObjectLifetimeScope.Application)
+        protected ManyObjectsResolverBase(ObjectLifetimeScope scope = ObjectLifetimeScope.Application)
             : this(new ActivatorServiceProvider(), LoggerResolver.Current.Logger, scope)
-		{
-			
-		}
+        {
+
+        }
 
         /// <summary>
         /// Initializes a new instance of the <see cref="ManyObjectsResolverBase{TResolver, TResolved}"/> class with an empty list of objects,
@@ -76,19 +76,19 @@ namespace Umbraco.Core.ObjectResolution
         /// <param name="httpContext">The HttpContextBase corresponding to the HttpRequest.</param>
         /// <exception cref="ArgumentNullException"><paramref name="httpContext"/> is <c>null</c>.</exception>
         protected ManyObjectsResolverBase(IServiceProvider serviceProvider, ILogger logger, HttpContextBase httpContext)
-		{
+        {
             if (serviceProvider == null) throw new ArgumentNullException("serviceProvider");
             if (httpContext == null) throw new ArgumentNullException("httpContext");
             CanResolveBeforeFrozen = false;
             Logger = logger;
-			LifetimeScope = ObjectLifetimeScope.HttpRequest;
-			_httpContextKey = GetType().FullName;
+            LifetimeScope = ObjectLifetimeScope.HttpRequest;
+            _httpContextKey = GetType().FullName;
             ServiceProvider = serviceProvider;
             CurrentHttpContext = httpContext;
-			_instanceTypes = new List<Type>();
+            _instanceTypes = new List<Type>();
 
             InitializeAppInstances();
-		}
+        }
 
         [EditorBrowsable(EditorBrowsableState.Never)]
         [Obsolete("Use ctor specifying IServiceProvider instead")]
@@ -110,57 +110,57 @@ namespace Umbraco.Core.ObjectResolution
         /// <exception cref="InvalidOperationException"><paramref name="scope"/> is per HttpRequest but the current HttpContext is null.</exception>
         protected ManyObjectsResolverBase(IServiceProvider serviceProvider, ILogger logger, IEnumerable<Type> value, ObjectLifetimeScope scope = ObjectLifetimeScope.Application)
             : this(serviceProvider, logger, scope)
-		{
-			_instanceTypes = value.ToList();
-		}
+        {
+            _instanceTypes = value.ToList();
+        }
 
         [EditorBrowsable(EditorBrowsableState.Never)]
         [Obsolete("Use ctor specifying IServiceProvider instead")]
         protected ManyObjectsResolverBase(IEnumerable<Type> value, ObjectLifetimeScope scope = ObjectLifetimeScope.Application)
             : this(new ActivatorServiceProvider(), LoggerResolver.Current.Logger, value, scope)
         {
-            
+
         }
 
-		/// <summary>
-		/// Initializes a new instance of the <see cref="ManyObjectsResolverBase{TResolver, TResolved}"/> class with an initial list of objects,
-		/// with creation of objects based on an HttpRequest lifetime scope.
-		/// </summary>
-		/// <param name="httpContext">The HttpContextBase corresponding to the HttpRequest.</param>
-		/// <param name="value">The list of object types.</param>
-		/// <exception cref="ArgumentNullException"><paramref name="httpContext"/> is <c>null</c>.</exception>
+        /// <summary>
+        /// Initializes a new instance of the <see cref="ManyObjectsResolverBase{TResolver, TResolved}"/> class with an initial list of objects,
+        /// with creation of objects based on an HttpRequest lifetime scope.
+        /// </summary>
+        /// <param name="httpContext">The HttpContextBase corresponding to the HttpRequest.</param>
+        /// <param name="value">The list of object types.</param>
+        /// <exception cref="ArgumentNullException"><paramref name="httpContext"/> is <c>null</c>.</exception>
         [Obsolete("Use ctor specifying IServiceProvider instead")]
         protected ManyObjectsResolverBase(HttpContextBase httpContext, IEnumerable<Type> value)
             : this(new ActivatorServiceProvider(), LoggerResolver.Current.Logger, httpContext)
-		{
-			_instanceTypes = value.ToList();
-		} 
-		#endregion
+        {
+            _instanceTypes = value.ToList();
+        }
+        #endregion
 
         private void InitializeAppInstances()
         {
             _applicationInstances = new Lazy<IEnumerable<TResolved>>(() => CreateInstances().ToArray());
         }
 
-		/// <summary>
-		/// Gets or sets a value indicating whether the resolver can resolve objects before resolution is frozen.
-		/// </summary>
-		/// <remarks>This is false by default and is used for some special internal resolvers.</remarks>
-		internal bool CanResolveBeforeFrozen { get; set; }
+        /// <summary>
+        /// Gets or sets a value indicating whether the resolver can resolve objects before resolution is frozen.
+        /// </summary>
+        /// <remarks>This is false by default and is used for some special internal resolvers.</remarks>
+        internal bool CanResolveBeforeFrozen { get; set; }
 
-		/// <summary>
-		/// Gets the list of types to create instances from.
-		/// </summary>
-		protected virtual IEnumerable<Type> InstanceTypes
-		{
-			get { return _instanceTypes; }
-		}
+        /// <summary>
+        /// Gets the list of types to create instances from.
+        /// </summary>
+        protected virtual IEnumerable<Type> InstanceTypes
+        {
+            get { return _instanceTypes; }
+        }
 
-		/// <summary>
-		/// Gets or sets the <see cref="HttpContextBase"/> used to initialize this object, if any.
-		/// </summary>
-		/// <remarks>If not null, then <c>LifetimeScope</c> will be <c>ObjectLifetimeScope.HttpRequest</c>.</remarks>
-		protected HttpContextBase CurrentHttpContext { get; private set; }
+        /// <summary>
+        /// Gets or sets the <see cref="HttpContextBase"/> used to initialize this object, if any.
+        /// </summary>
+        /// <remarks>If not null, then <c>LifetimeScope</c> will be <c>ObjectLifetimeScope.HttpRequest</c>.</remarks>
+        protected HttpContextBase CurrentHttpContext { get; private set; }
 
         /// <summary>
         /// Returns the service provider used to instantiate objects
@@ -174,16 +174,16 @@ namespace Umbraco.Core.ObjectResolution
 		/// </summary>
 		protected ObjectLifetimeScope LifetimeScope { get; private set; }
 
-		/// <summary>
-		/// Gets the resolved object instances, sorted by weight.
-		/// </summary>
-		/// <returns>The sorted resolved object instances.</returns>
-		/// <remarks>
-		/// <para>The order is based upon the <c>WeightedPluginAttribute</c> and <c>DefaultPluginWeight</c>.</para>
-		/// <para>Weights are sorted ascendingly (lowest weights come first).</para>
-		/// </remarks>
-		protected IEnumerable<TResolved> GetSortedValues()
-		{
+        /// <summary>
+        /// Gets the resolved object instances, sorted by weight.
+        /// </summary>
+        /// <returns>The sorted resolved object instances.</returns>
+        /// <remarks>
+        /// <para>The order is based upon the <c>WeightAttribute</c> and <c>DefaultPluginWeight</c>.</para>
+        /// <para>Weights are sorted ascendingly (lowest weights come first).</para>
+        /// </remarks>
+        protected IEnumerable<TResolved> GetSortedValues()
+        {
             if (_sortedValues == null)
             {
                 var values = Values.ToList();
@@ -191,18 +191,18 @@ namespace Umbraco.Core.ObjectResolution
                 _sortedValues = values;
             }
             return _sortedValues;
-		}
+        }
 
-		/// <summary>
-		/// Gets or sets the default type weight.
-		/// </summary>
-		/// <remarks>Determines the weight of types that do not have a <c>WeightedPluginAttribute</c> set on 
-		/// them, when calling <c>GetSortedValues</c>.</remarks>
-		protected virtual int DefaultPluginWeight
-		{
-			get { return _defaultPluginWeight; }
-			set { _defaultPluginWeight = value; }
-		}
+        /// <summary>
+        /// Gets or sets the default type weight.
+        /// </summary>
+        /// <remarks>Determines the weight of types that do not have a <c>WeightAttribute</c> set on 
+        /// them, when calling <c>GetSortedValues</c>.</remarks>
+        protected virtual int DefaultPluginWeight
+        {
+            get { return _defaultPluginWeight; }
+            set { _defaultPluginWeight = value; }
+        }
 
         /// <summary>
         /// Returns the weight of an object for user with GetSortedValues
@@ -210,22 +210,22 @@ namespace Umbraco.Core.ObjectResolution
         /// <param name="o"></param>
         /// <returns></returns>
 		protected virtual int GetObjectWeight(object o)
-		{
-			var type = o.GetType();
-			var attr = type.GetCustomAttribute<WeightedPluginAttribute>(true);
-			return attr == null ? DefaultPluginWeight : attr.Weight;
-		}
+        {
+            var type = o.GetType();
+            var attr = type.GetCustomAttribute<WeightAttribute>(true);
+            return attr == null ? DefaultPluginWeight : attr.Weight;
+        }
 
-		/// <summary>
-		/// Gets the resolved object instances.
-		/// </summary>
-		/// <exception cref="InvalidOperationException"><c>CanResolveBeforeFrozen</c> is false, and resolution is not frozen.</exception>
-		protected IEnumerable<TResolved> Values
-		{
-			get
-			{
-			    using (Resolution.Reader(CanResolveBeforeFrozen))
-			    {
+        /// <summary>
+        /// Gets the resolved object instances.
+        /// </summary>
+        /// <exception cref="InvalidOperationException"><c>CanResolveBeforeFrozen</c> is false, and resolution is not frozen.</exception>
+        protected IEnumerable<TResolved> Values
+        {
+            get
+            {
+                using (Resolution.Reader(CanResolveBeforeFrozen))
+                {
                     // note: we apply .ToArray() to the output of CreateInstance() because that is an IEnumerable that
                     // comes from the PluginManager we want to be _sure_ that it's not a Linq of some sort, but the
                     // instances have actually been instanciated when we return.
@@ -247,7 +247,7 @@ namespace Umbraco.Core.ObjectResolution
                                 CurrentHttpContext.Items[_httpContextKey] = instances;
                             }
                             return (TResolved[])CurrentHttpContext.Items[_httpContextKey];
-                            
+
                         case ObjectLifetimeScope.Application:
 
                             return _applicationInstances.Value;
@@ -258,132 +258,132 @@ namespace Umbraco.Core.ObjectResolution
                             return CreateInstances().ToArray();
                     }
                 }
-			}
-		}
+            }
+        }
 
-		/// <summary>
-		/// Creates the object instances for the types contained in the types collection.
-		/// </summary>
-		/// <returns>A list of objects of type <typeparamref name="TResolved"/>.</returns>
-		protected virtual IEnumerable<TResolved> CreateInstances()
-		{
-			return ServiceProvider.CreateInstances<TResolved>(InstanceTypes, Logger);
-		}
+        /// <summary>
+        /// Creates the object instances for the types contained in the types collection.
+        /// </summary>
+        /// <returns>A list of objects of type <typeparamref name="TResolved"/>.</returns>
+        protected virtual IEnumerable<TResolved> CreateInstances()
+        {
+            return ServiceProvider.CreateInstances<TResolved>(InstanceTypes, Logger);
+        }
 
-		#region Types collection manipulation
+        #region Types collection manipulation
 
-		/// <summary>
-		/// Removes a type.
-		/// </summary>
-		/// <param name="value">The type to remove.</param>
-		/// <exception cref="InvalidOperationException">the resolver does not support removing types, or 
-		/// the type is not a valid type for the resolver.</exception>
-		public virtual void RemoveType(Type value)
-		{
-			EnsureSupportsRemove();
+        /// <summary>
+        /// Removes a type.
+        /// </summary>
+        /// <param name="value">The type to remove.</param>
+        /// <exception cref="InvalidOperationException">the resolver does not support removing types, or 
+        /// the type is not a valid type for the resolver.</exception>
+        public virtual void RemoveType(Type value)
+        {
+            EnsureSupportsRemove();
 
-			using (Resolution.Configuration)
-			using (var l = new UpgradeableReadLock(_lock))
-			{
-				EnsureCorrectType(value);
+            using (Resolution.Configuration)
+            using (var l = new UpgradeableReadLock(_lock))
+            {
+                EnsureCorrectType(value);
 
-				l.UpgradeToWriteLock();
-				_instanceTypes.Remove(value);
-			}
-		}
+                l.UpgradeToWriteLock();
+                _instanceTypes.Remove(value);
+            }
+        }
 
-		/// <summary>
-		/// Removes a type.
-		/// </summary>
-		/// <typeparam name="T">The type to remove.</typeparam>
-		/// <exception cref="InvalidOperationException">the resolver does not support removing types, or 
-		/// the type is not a valid type for the resolver.</exception>
-		public void RemoveType<T>()
+        /// <summary>
+        /// Removes a type.
+        /// </summary>
+        /// <typeparam name="T">The type to remove.</typeparam>
+        /// <exception cref="InvalidOperationException">the resolver does not support removing types, or 
+        /// the type is not a valid type for the resolver.</exception>
+        public void RemoveType<T>()
             where T : TResolved
-		{
-			RemoveType(typeof(T));
-		}
+        {
+            RemoveType(typeof(T));
+        }
 
-		/// <summary>
-		/// Adds types.
-		/// </summary>
-		/// <param name="types">The types to add.</param>
-		/// <remarks>The types are appended at the end of the list.</remarks>
-		/// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
-		/// a type is not a valid type for the resolver, or a type is already in the collection of types.</exception>
-		protected void AddTypes(IEnumerable<Type> types)
-		{
-			EnsureSupportsAdd();
+        /// <summary>
+        /// Adds types.
+        /// </summary>
+        /// <param name="types">The types to add.</param>
+        /// <remarks>The types are appended at the end of the list.</remarks>
+        /// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
+        /// a type is not a valid type for the resolver, or a type is already in the collection of types.</exception>
+        protected void AddTypes(IEnumerable<Type> types)
+        {
+            EnsureSupportsAdd();
 
-			using (Resolution.Configuration)
-			using (new WriteLock(_lock))
-			{
-				foreach(var t in types)
-				{
-					EnsureCorrectType(t);
+            using (Resolution.Configuration)
+            using (new WriteLock(_lock))
+            {
+                foreach (var t in types)
+                {
+                    EnsureCorrectType(t);
                     if (_instanceTypes.Contains(t))
-					{
-						throw new InvalidOperationException(string.Format(
-							"Type {0} is already in the collection of types.", t.FullName));
-					}
-					_instanceTypes.Add(t);	
-				}				
-			}
-		}
+                    {
+                        throw new InvalidOperationException(string.Format(
+                            "Type {0} is already in the collection of types.", t.FullName));
+                    }
+                    _instanceTypes.Add(t);
+                }
+            }
+        }
 
-		/// <summary>
-		/// Adds a type.
-		/// </summary>
-		/// <param name="value">The type to add.</param>
-		/// <remarks>The type is appended at the end of the list.</remarks>
-		/// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
-		/// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
-		public virtual void AddType(Type value)
-		{
-			EnsureSupportsAdd();
+        /// <summary>
+        /// Adds a type.
+        /// </summary>
+        /// <param name="value">The type to add.</param>
+        /// <remarks>The type is appended at the end of the list.</remarks>
+        /// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
+        /// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
+        public virtual void AddType(Type value)
+        {
+            EnsureSupportsAdd();
 
-			using (Resolution.Configuration)
-			using (var l = new UpgradeableReadLock(_lock))
-			{
-				EnsureCorrectType(value);
+            using (Resolution.Configuration)
+            using (var l = new UpgradeableReadLock(_lock))
+            {
+                EnsureCorrectType(value);
                 if (_instanceTypes.Contains(value))
-				{
-					throw new InvalidOperationException(string.Format(
-						"Type {0} is already in the collection of types.", value.FullName));
-				}
+                {
+                    throw new InvalidOperationException(string.Format(
+                        "Type {0} is already in the collection of types.", value.FullName));
+                }
 
-				l.UpgradeToWriteLock();
-				_instanceTypes.Add(value);
-			}
-		}
+                l.UpgradeToWriteLock();
+                _instanceTypes.Add(value);
+            }
+        }
 
-		/// <summary>
-		/// Adds a type.
-		/// </summary>
-		/// <typeparam name="T">The type to add.</typeparam>
-		/// <remarks>The type is appended at the end of the list.</remarks>
-		/// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
-		/// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
-		public void AddType<T>()
+        /// <summary>
+        /// Adds a type.
+        /// </summary>
+        /// <typeparam name="T">The type to add.</typeparam>
+        /// <remarks>The type is appended at the end of the list.</remarks>
+        /// <exception cref="InvalidOperationException">the resolver does not support adding types, or 
+        /// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
+        public void AddType<T>()
             where T : TResolved
-		{
-			AddType(typeof(T));
-		}
+        {
+            AddType(typeof(T));
+        }
 
-		/// <summary>
-		/// Clears the list of types
-		/// </summary>
-		/// <exception cref="InvalidOperationException">the resolver does not support clearing types.</exception>
-		public virtual void Clear()
-		{
-			EnsureSupportsClear();
+        /// <summary>
+        /// Clears the list of types
+        /// </summary>
+        /// <exception cref="InvalidOperationException">the resolver does not support clearing types.</exception>
+        public virtual void Clear()
+        {
+            EnsureSupportsClear();
 
-			using (Resolution.Configuration)
-			using (new WriteLock(_lock))
-			{
-				_instanceTypes.Clear();
-			}
-		}
+            using (Resolution.Configuration)
+            using (new WriteLock(_lock))
+            {
+                _instanceTypes.Clear();
+            }
+        }
 
         /// <summary>
         /// WARNING! Do not use this unless you know what you are doing, clear all types registered and instances
@@ -399,32 +399,32 @@ namespace Umbraco.Core.ObjectResolution
             }
         }
 
-		/// <summary>
-		/// Inserts a type at the specified index.
-		/// </summary>
-		/// <param name="index">The zero-based index at which the type should be inserted.</param>
-		/// <param name="value">The type to insert.</param>
-		/// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
-		/// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
-		/// <exception cref="ArgumentOutOfRangeException"><paramref name="index"/> is out of range.</exception>
-		public virtual void InsertType(int index, Type value)
-		{			
-			EnsureSupportsInsert();
+        /// <summary>
+        /// Inserts a type at the specified index.
+        /// </summary>
+        /// <param name="index">The zero-based index at which the type should be inserted.</param>
+        /// <param name="value">The type to insert.</param>
+        /// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
+        /// the type is not a valid type for the resolver, or the type is already in the collection of types.</exception>
+        /// <exception cref="ArgumentOutOfRangeException"><paramref name="index"/> is out of range.</exception>
+        public virtual void InsertType(int index, Type value)
+        {
+            EnsureSupportsInsert();
 
-			using (Resolution.Configuration)
-			using (var l = new UpgradeableReadLock(_lock))
-			{
-				EnsureCorrectType(value);
+            using (Resolution.Configuration)
+            using (var l = new UpgradeableReadLock(_lock))
+            {
+                EnsureCorrectType(value);
                 if (_instanceTypes.Contains(value))
-				{
-					throw new InvalidOperationException(string.Format(
-						"Type {0} is already in the collection of types.", value.FullName));
-				}
+                {
+                    throw new InvalidOperationException(string.Format(
+                        "Type {0} is already in the collection of types.", value.FullName));
+                }
 
-				l.UpgradeToWriteLock();
-				_instanceTypes.Insert(index, value);
-			}
-		}
+                l.UpgradeToWriteLock();
+                _instanceTypes.Insert(index, value);
+            }
+        }
 
         /// <summary>
         /// Inserts a type at the beginning of the list.
@@ -445,9 +445,9 @@ namespace Umbraco.Core.ObjectResolution
 		/// <exception cref="ArgumentOutOfRangeException"><paramref name="index"/> is out of range.</exception>
 		public void InsertType<T>(int index)
             where T : TResolved
-		{
-			InsertType(index, typeof(T));
-		}
+        {
+            InsertType(index, typeof(T));
+        }
 
         /// <summary>
         /// Inserts a type at the beginning of the list.
@@ -458,68 +458,68 @@ namespace Umbraco.Core.ObjectResolution
         {
             InsertType(0, typeof(T));
         }
-        
-        /// <summary>
-		/// Inserts a type before a specified, already existing type.
-		/// </summary>
-		/// <param name="existingType">The existing type before which to insert.</param>
-		/// <param name="value">The type to insert.</param>
-		/// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
-		/// one of the types is not a valid type for the resolver, or the existing type is not in the collection,
-		/// or the new type is already in the collection of types.</exception>
-		public virtual void InsertTypeBefore(Type existingType, Type value)
-		{
-			EnsureSupportsInsert();
 
-			using (Resolution.Configuration)
-			using (var l = new UpgradeableReadLock(_lock))
-			{
-				EnsureCorrectType(existingType);
-				EnsureCorrectType(value);
+        /// <summary>
+        /// Inserts a type before a specified, already existing type.
+        /// </summary>
+        /// <param name="existingType">The existing type before which to insert.</param>
+        /// <param name="value">The type to insert.</param>
+        /// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
+        /// one of the types is not a valid type for the resolver, or the existing type is not in the collection,
+        /// or the new type is already in the collection of types.</exception>
+        public virtual void InsertTypeBefore(Type existingType, Type value)
+        {
+            EnsureSupportsInsert();
+
+            using (Resolution.Configuration)
+            using (var l = new UpgradeableReadLock(_lock))
+            {
+                EnsureCorrectType(existingType);
+                EnsureCorrectType(value);
                 if (_instanceTypes.Contains(existingType) == false)
-				{
-					throw new InvalidOperationException(string.Format(
-						"Type {0} is not in the collection of types.", existingType.FullName));
-				}
+                {
+                    throw new InvalidOperationException(string.Format(
+                        "Type {0} is not in the collection of types.", existingType.FullName));
+                }
                 if (_instanceTypes.Contains(value))
-				{
-					throw new InvalidOperationException(string.Format(
-						"Type {0} is already in the collection of types.", value.FullName));
-				}
+                {
+                    throw new InvalidOperationException(string.Format(
+                        "Type {0} is already in the collection of types.", value.FullName));
+                }
                 int index = _instanceTypes.IndexOf(existingType);
 
-				l.UpgradeToWriteLock();
-				_instanceTypes.Insert(index, value);
-			}
-		}
+                l.UpgradeToWriteLock();
+                _instanceTypes.Insert(index, value);
+            }
+        }
 
-		/// <summary>
-		/// Inserts a type before a specified, already existing type.
-		/// </summary>
-		/// <typeparam name="TExisting">The existing type before which to insert.</typeparam>
-		/// <typeparam name="T">The type to insert.</typeparam>
-		/// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
-		/// one of the types is not a valid type for the resolver, or the existing type is not in the collection,
-		/// or the new type is already in the collection of types.</exception>
-		public void InsertTypeBefore<TExisting, T>()
+        /// <summary>
+        /// Inserts a type before a specified, already existing type.
+        /// </summary>
+        /// <typeparam name="TExisting">The existing type before which to insert.</typeparam>
+        /// <typeparam name="T">The type to insert.</typeparam>
+        /// <exception cref="InvalidOperationException">the resolver does not support inserting types, or 
+        /// one of the types is not a valid type for the resolver, or the existing type is not in the collection,
+        /// or the new type is already in the collection of types.</exception>
+        public void InsertTypeBefore<TExisting, T>()
             where TExisting : TResolved
             where T : TResolved
-		{
-			InsertTypeBefore(typeof(TExisting), typeof(T));
-		}
+        {
+            InsertTypeBefore(typeof(TExisting), typeof(T));
+        }
 
-		/// <summary>
-		/// Returns a value indicating whether the specified type is already in the collection of types.
-		/// </summary>
-		/// <param name="value">The type to look for.</param>
-		/// <returns>A value indicating whether the type is already in the collection of types.</returns>
-		public virtual bool ContainsType(Type value)
-		{
-			using (new ReadLock(_lock))
-			{
-				return _instanceTypes.Contains(value);
-			}
-		}
+        /// <summary>
+        /// Returns a value indicating whether the specified type is already in the collection of types.
+        /// </summary>
+        /// <param name="value">The type to look for.</param>
+        /// <returns>A value indicating whether the type is already in the collection of types.</returns>
+        public virtual bool ContainsType(Type value)
+        {
+            using (new ReadLock(_lock))
+            {
+                return _instanceTypes.Contains(value);
+            }
+        }
 
         /// <summary>
         /// Gets the types in the collection of types.
@@ -536,27 +536,27 @@ namespace Umbraco.Core.ObjectResolution
             return types;
         }
 
-		/// <summary>
-		/// Returns a value indicating whether the specified type is already in the collection of types.
-		/// </summary>
-		/// <typeparam name="T">The type to look for.</typeparam>
-		/// <returns>A value indicating whether the type is already in the collection of types.</returns>
-		public bool ContainsType<T>()
+        /// <summary>
+        /// Returns a value indicating whether the specified type is already in the collection of types.
+        /// </summary>
+        /// <typeparam name="T">The type to look for.</typeparam>
+        /// <returns>A value indicating whether the type is already in the collection of types.</returns>
+        public bool ContainsType<T>()
             where T : TResolved
-		{
-			return ContainsType(typeof(T));
-		}
+        {
+            return ContainsType(typeof(T));
+        }
 
-		#endregion
+        #endregion
 
-		/// <summary>
-		/// Returns a WriteLock to use when modifying collections
-		/// </summary>
-		/// <returns></returns>
-		protected WriteLock GetWriteLock()
-		{
-			return new WriteLock(_lock);
-		}
+        /// <summary>
+        /// Returns a WriteLock to use when modifying collections
+        /// </summary>
+        /// <returns></returns>
+        protected WriteLock GetWriteLock()
+        {
+            return new WriteLock(_lock);
+        }
 
         #region Type utilities
 
@@ -581,70 +581,71 @@ namespace Umbraco.Core.ObjectResolution
         /// </summary>
         /// <exception cref="InvalidOperationException">The resolver does not support removing types.</exception>
         protected void EnsureSupportsRemove()
-		{
-			if (SupportsRemove == false)
+        {
+            if (SupportsRemove == false)
                 throw new InvalidOperationException("This resolver does not support removing types");
-		}
+        }
 
         /// <summary>
         /// Ensures that the resolver supports clearing types.
         /// </summary>
         /// <exception cref="InvalidOperationException">The resolver does not support clearing types.</exception>
-        protected void EnsureSupportsClear()		{
-			if (SupportsClear == false)
+        protected void EnsureSupportsClear()
+        {
+            if (SupportsClear == false)
                 throw new InvalidOperationException("This resolver does not support clearing types");
-		}
+        }
 
         /// <summary>
         /// Ensures that the resolver supports adding types.
         /// </summary>
         /// <exception cref="InvalidOperationException">The resolver does not support adding types.</exception>
         protected void EnsureSupportsAdd()
-		{
-			if (SupportsAdd == false)
+        {
+            if (SupportsAdd == false)
                 throw new InvalidOperationException("This resolver does not support adding new types");
-		}
+        }
 
         /// <summary>
         /// Ensures that the resolver supports inserting types.
         /// </summary>
         /// <exception cref="InvalidOperationException">The resolver does not support inserting types.</exception>
         protected void EnsureSupportsInsert()
-		{
-			if (SupportsInsert == false)
+        {
+            if (SupportsInsert == false)
                 throw new InvalidOperationException("This resolver does not support inserting new types");
-		}
+        }
 
         /// <summary>
         /// Gets a value indicating whether the resolver supports adding types.
         /// </summary>
 		protected virtual bool SupportsAdd
-		{
-			get { return true; }
-		}
+        {
+            get { return true; }
+        }
 
         /// <summary>
         /// Gets a value indicating whether the resolver supports inserting types.
         /// </summary>
         protected virtual bool SupportsInsert
-		{
-			get { return true; }
-		}
+        {
+            get { return true; }
+        }
 
         /// <summary>
         /// Gets a value indicating whether the resolver supports clearing types.
         /// </summary>
         protected virtual bool SupportsClear
-		{
-			get { return true; }
-		}
+        {
+            get { return true; }
+        }
 
         /// <summary>
         /// Gets a value indicating whether the resolver supports removing types.
         /// </summary>
         protected virtual bool SupportsRemove
-		{
-			get { return true; }
+        {
+            get { return true; }
         }
 
         #endregion

src/Umbraco.Core/ObjectResolution/WeightAttribute.cs

@@ -1,25 +1,25 @@
-using System;
-
-namespace Umbraco.Core.ObjectResolution
-{
-	/// <summary>
-	/// Indicates the relative weight of a resolved object type.
-	/// </summary>
-	[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
-	internal class WeightedPluginAttribute : Attribute
-	{
-		/// <summary>
-		/// Initializes a new instance of the <see cref="WeightedPluginAttribute"/> class with a weight.
-		/// </summary>
-		/// <param name="weight">The object type weight.</param>
-		public WeightedPluginAttribute(int weight)
-		{
-			Weight = weight;
-		}
-
-		/// <summary>
-		/// Gets or sets the weight of the object type.
-		/// </summary>
-		public int Weight { get; private set; }
-	}
+using System;
+
+namespace Umbraco.Core.ObjectResolution
+{
+	/// <summary>
+	/// Indicates the relative weight of a resolved object type.
+	/// </summary>
+	[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)]
+	public class WeightAttribute : Attribute
+	{
+        /// <summary>
+		/// Initializes a new instance of the <see cref="WeightAttribute"/> class with a weight.
+		/// </summary>
+		/// <param name="weight">The object type weight.</param>
+	    public WeightAttribute(int weight)
+	    {
+	        Weight = weight;
+	    }
+
+		/// <summary>
+		/// Gets or sets the weight of the object type.
+		/// </summary>
+		public int Weight { get; private set; }
+	}
 }

src/Umbraco.Core/Persistence/DatabaseModelDefinitions/DefinitionFactory.cs

@@ -9,7 +9,7 @@ namespace Umbraco.Core.Persistence.DatabaseModelDefinitions
 {
     internal static class DefinitionFactory
     {
-        public static TableDefinition GetTableDefinition(Type modelType)
+        public static TableDefinition GetTableDefinition(ISqlSyntaxProvider syntaxProvider, Type modelType)
         {
             //Looks for PetaPoco's TableNameAtribute for the name of the table
             //If no attribute is set we use the name of the Type as the default convention
@@ -32,7 +32,7 @@ namespace Umbraco.Core.Persistence.DatabaseModelDefinitions
                 //Otherwise use the name of the property itself as the default convention
                 var columnAttribute = propertyInfo.FirstAttribute<ColumnAttribute>();
                 string columnName = columnAttribute != null ? columnAttribute.Name : propertyInfo.Name;
-                var columnDefinition = GetColumnDefinition(modelType, propertyInfo, columnName, tableName);
+                var columnDefinition = GetColumnDefinition(syntaxProvider, modelType, propertyInfo, columnName, tableName);
                 tableDefinition.Columns.Add(columnDefinition);
 
                 //Creates a foreignkey definition and adds it to the collection on the table definition
@@ -58,7 +58,7 @@ namespace Umbraco.Core.Persistence.DatabaseModelDefinitions
             return tableDefinition;
         }
 
-        public static ColumnDefinition GetColumnDefinition(Type modelType, PropertyInfo propertyInfo, string columnName, string tableName)
+        public static ColumnDefinition GetColumnDefinition(ISqlSyntaxProvider syntaxProvider, Type modelType, PropertyInfo propertyInfo, string columnName, string tableName)
         {
             var definition = new ColumnDefinition{ Name = columnName, TableName = tableName, ModificationType = ModificationType.Create };
 
@@ -110,7 +110,7 @@ namespace Umbraco.Core.Persistence.DatabaseModelDefinitions
             {
                 //Special case for MySQL as it can't have multiple default DateTime values, which 
                 //is what the umbracoServer table definition is trying to create
-                if (SqlSyntaxContext.SqlSyntaxProvider is MySqlSyntaxProvider && definition.TableName == "umbracoServer" &&
+                if (syntaxProvider is MySqlSyntaxProvider && definition.TableName == "umbracoServer" &&
                         definition.TableName.ToLowerInvariant() == "lastNotifiedDate".ToLowerInvariant())
                     return definition;
 

src/Umbraco.Core/Persistence/DatabaseSchemaHelper.cs

@@ -99,7 +99,7 @@ namespace Umbraco.Core.Persistence
 
         public void CreateTable(bool overwrite, Type modelType)
         {
-            var tableDefinition = DefinitionFactory.GetTableDefinition(modelType);
+            var tableDefinition = DefinitionFactory.GetTableDefinition(_syntaxProvider, modelType);
             var tableName = tableDefinition.Name;
 
             string createSql = _syntaxProvider.Format(tableDefinition);

src/Umbraco.Core/Persistence/DbConnectionExtensions.cs

@@ -4,9 +4,11 @@ using System.Configuration;
 using System.Data;
 using System.Data.Common;
 using System.Data.SqlClient;
+using System.Data.SqlServerCe;
 using System.Linq;
 using System.Text;
 using System.Threading.Tasks;
+using MySql.Data.MySqlClient;
 using Umbraco.Core.Logging; 
 
 namespace Umbraco.Core.Persistence
@@ -69,6 +71,38 @@ namespace Umbraco.Core.Persistence
             }
         }
 
+        public static string GetConnStringExSecurityInfo(this IDbConnection connection)
+        {
+            try
+            {
+                if (connection is SqlConnection)
+                {
+                    var builder = new SqlConnectionStringBuilder(connection.ConnectionString);
+                    return string.Format("DataSource: {0}, InitialCatalog: {1}", builder.DataSource, builder.InitialCatalog);
+                }
+
+                if (connection is SqlCeConnection)
+                {
+                    var builder = new SqlCeConnectionStringBuilder(connection.ConnectionString);
+                    return string.Format("DataSource: {0}", builder.DataSource);
+                }
+
+                if (connection is MySqlConnection)
+                {
+                    var builder = new MySqlConnectionStringBuilder(connection.ConnectionString);
+                    return string.Format("Server: {0}, Database: {1}", builder.Server, builder.Database);
+                }
+            }
+            catch (Exception ex)
+            {
+                LogHelper.WarnWithException(typeof(DbConnectionExtensions),
+                    "Could not resolve connection string parameters", ex);
+                return "(Could not resolve)";
+            }
+
+            throw new ArgumentException(string.Format("The connection type {0} is not supported", connection.GetType()));
+        }
+
         public static bool IsAvailable(this IDbConnection connection)
         {
             try
@@ -79,7 +113,8 @@ namespace Umbraco.Core.Persistence
             catch (DbException exc)
             {
                 // Don't swallow this error, the exception is super handy for knowing "why" its not available
-                LogHelper.WarnWithException<IDbConnection>("Configured database is reporting as not being available!", exc);
+                LogHelper.WarnWithException(typeof(DbConnectionExtensions),
+                    "Configured database is reporting as not being available! {0}", exc, connection.GetConnStringExSecurityInfo);
                 return false;
             }
 

src/Umbraco.Core/Persistence/Mappers/BaseMapper.cs

@@ -8,7 +8,17 @@ namespace Umbraco.Core.Persistence.Mappers
 {
     public abstract class BaseMapper
     {
-        
+        private readonly ISqlSyntaxProvider _sqlSyntax;
+
+        protected BaseMapper() : this(SqlSyntaxContext.SqlSyntaxProvider)
+        {            
+        }
+
+        protected BaseMapper(ISqlSyntaxProvider sqlSyntax)
+        {
+            _sqlSyntax = sqlSyntax;
+        }
+
         internal abstract ConcurrentDictionary<string, DtoMapModel> PropertyInfoCache { get; }
 
         internal abstract void BuildMap();
@@ -58,8 +68,8 @@ namespace Umbraco.Core.Persistence.Mappers
             string columnName = columnAttribute.Name;
 
             string columnMap = string.Format("{0}.{1}",
-                                             SqlSyntaxContext.SqlSyntaxProvider.GetQuotedTableName(tableName),
-                                             SqlSyntaxContext.SqlSyntaxProvider.GetQuotedColumnName(columnName));
+                                             _sqlSyntax.GetQuotedTableName(tableName),
+                                             _sqlSyntax.GetQuotedColumnName(columnName));
             return columnMap;
         }
     }

src/Umbraco.Core/Persistence/Mappers/ContentMapper.cs

@@ -3,6 +3,7 @@ using System.Collections.Concurrent;
 using System.Linq.Expressions;
 using Umbraco.Core.Models;
 using Umbraco.Core.Models.Rdbms;
+using Umbraco.Core.Persistence.SqlSyntax;
 
 namespace Umbraco.Core.Persistence.Mappers
 {
@@ -16,6 +17,11 @@ namespace Umbraco.Core.Persistence.Mappers
     {
         private static readonly ConcurrentDictionary<string, DtoMapModel> PropertyInfoCacheInstance = new ConcurrentDictionary<string, DtoMapModel>();
 
+        public ContentMapper(ISqlSyntaxProvider sqlSyntax) : base(sqlSyntax)
+        {
+            
+        }
+
         //NOTE: its an internal class but the ctor must be public since we're using Activator.CreateInstance to create it
         // otherwise that would fail because there is no public constructor.
         public ContentMapper()

src/Umbraco.Core/Persistence/Mappers/MappingResolver.cs

@@ -43,7 +43,7 @@ namespace Umbraco.Core.Persistence.Mappers
                     {
                         return byAttribute.Result;
                     }
-                    throw new Exception("Invalid Type: A Mapper could not be resolved based on the passed in Type");
+                    throw new Exception("Invalid Type: A Mapper could not be resolved based on the passed in Type " + type);
                 });
         }
 

src/Umbraco.Core/Persistence/Migrations/Initial/BaseDataCreation.cs

@@ -149,8 +149,8 @@ namespace Umbraco.Core.Persistence.Migrations.Initial
         private void CreateCmsContentTypeData()
         {
             _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 532, NodeId = 1031, Alias = Constants.Conventions.MediaTypes.Folder, Icon = "icon-folder", Thumbnail = "icon-folder", IsContainer = false, AllowAtRoot = true });
-            _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 533, NodeId = 1032, Alias = Constants.Conventions.MediaTypes.Image, Icon = "icon-picture", Thumbnail = "icon-picture" });
-            _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 534, NodeId = 1033, Alias = Constants.Conventions.MediaTypes.File, Icon = "icon-document", Thumbnail = "icon-document" });
+            _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 533, NodeId = 1032, Alias = Constants.Conventions.MediaTypes.Image, Icon = "icon-picture", Thumbnail = "icon-picture", AllowAtRoot = true });
+            _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 534, NodeId = 1033, Alias = Constants.Conventions.MediaTypes.File, Icon = "icon-document", Thumbnail = "icon-document", AllowAtRoot = true });
             _database.Insert("cmsContentType", "pk", false, new ContentTypeDto { PrimaryKey = 531, NodeId = 1044, Alias = Constants.Conventions.MemberTypes.DefaultAlias, Icon = "icon-user", Thumbnail = "icon-user" });
         }
 

src/Umbraco.Core/Persistence/Migrations/Initial/DatabaseSchemaCreation.cs

@@ -159,7 +159,7 @@ namespace Umbraco.Core.Persistence.Migrations.Initial
 
             foreach (var item in OrderedTables.OrderBy(x => x.Key))
             {
-                var tableDefinition = DefinitionFactory.GetTableDefinition(item.Value);
+                var tableDefinition = DefinitionFactory.GetTableDefinition(_sqlSyntaxProvider, item.Value);
                 result.TableDefinitions.Add(tableDefinition);
             }
 

src/Umbraco.Core/Persistence/Migrations/Upgrades/TargetVersionSevenFiveFive/UpdateAllowedMediaTypesAtRoot.cs

@@ -0,0 +1,25 @@
+using Umbraco.Core.Configuration;
+using Umbraco.Core.Logging;
+using Umbraco.Core.Persistence.SqlSyntax;
+
+namespace Umbraco.Core.Persistence.Migrations.Upgrades.TargetVersionSevenFiveFive
+{
+    /// <summary>
+    /// See: http://issues.umbraco.org/issue/U4-4196
+    /// </summary>
+    [Migration("7.5.5", 1, GlobalSettings.UmbracoMigrationName)]
+    public class UpdateAllowedMediaTypesAtRoot : MigrationBase
+    {
+        public UpdateAllowedMediaTypesAtRoot(ISqlSyntaxProvider sqlSyntax, ILogger logger)
+            : base(sqlSyntax, logger)
+        { }
+
+        public override void Up()
+        {
+            Execute.Sql("UPDATE cmsContentType SET allowAtRoot = 1 WHERE nodeId = 1032 OR nodeId = 1033");
+        }
+
+        public override void Down()
+        { }
+    }
+}

src/Umbraco.Core/Persistence/Migrations/Upgrades/TargetVersionSevenSixZero/RemovePropertyDataIdIndex.cs

@@ -0,0 +1,38 @@
+using System.Linq;
+using Umbraco.Core.Configuration;
+using Umbraco.Core.Logging;
+using Umbraco.Core.Persistence.SqlSyntax;
+
+namespace Umbraco.Core.Persistence.Migrations.Upgrades.TargetVersionSevenSixZero
+{
+    /// <summary>
+    /// See: http://issues.umbraco.org/issue/U4-9188
+    /// </summary>
+    [Migration("7.6.0", 0, GlobalSettings.UmbracoMigrationName)]
+    public class UpdateUniqueIndexOnCmsPropertyData : MigrationBase
+    {
+        public UpdateUniqueIndexOnCmsPropertyData(ISqlSyntaxProvider sqlSyntax, ILogger logger)
+            : base(sqlSyntax, logger)
+        {
+        }
+
+        public override void Up()
+        {            
+            //tuple = tablename, indexname, columnname, unique
+            var indexes = SqlSyntax.GetDefinedIndexes(Context.Database).ToArray();
+            var found = indexes.FirstOrDefault(
+                x => x.Item1.InvariantEquals("cmsPropertyData")
+                     && x.Item2.InvariantEquals("IX_cmsPropertyData"));
+
+            if (found != null)
+            {
+                //drop the index
+                Delete.Index("IX_cmsPropertyData").OnTable("cmsPropertyData");             
+            }
+        }
+
+        public override void Down()
+        {
+        }
+    }
+}

src/Umbraco.Core/Persistence/PetaPoco.cs

@@ -739,6 +739,11 @@ namespace Umbraco.Core.Persistence
 	    /// <param name="take"></param>
 	    internal virtual void BuildSqlDbSpecificPagingQuery(DBType databaseType, long skip, long take, string sql, string sqlSelectRemoved, string sqlOrderBy, ref object[] args, out string sqlPage)
 	    {
+            // this is overriden in UmbracoDatabase, and if running SqlServer >=2012, the database type
+            // is switched from SqlServer to SqlServerCE in order to use the better paging syntax that
+            // SqlCE supports, and SqlServer >=2012 too.
+            // so the first case is actually for SqlServer <2012, and second case is CE *and* SqlServer >=2012
+
             if (databaseType == DBType.SqlServer || databaseType == DBType.Oracle)
             {
                 sqlSelectRemoved = rxOrderBy.Replace(sqlSelectRemoved, "");
@@ -746,8 +751,16 @@ namespace Umbraco.Core.Persistence
                 {
                     sqlSelectRemoved = "peta_inner.* FROM (SELECT " + sqlSelectRemoved + ") peta_inner";
                 }
-                sqlPage = string.Format("SELECT * FROM (SELECT ROW_NUMBER() OVER ({0}) peta_rn, {1}) peta_paged WHERE peta_rn>@{2} AND peta_rn<=@{3}",
-                                        sqlOrderBy == null ? "ORDER BY (SELECT NULL)" : sqlOrderBy, sqlSelectRemoved, args.Length, args.Length + 1);
+
+                // split to ensure that peta_rn is the last field to be selected, else Page<int> would fail
+                // the resulting sql is not perfect, NPoco has a much nicer way to do it, but it would require
+                // importing large parts of NPoco
+                var pos = sqlSelectRemoved.IndexOf("FROM");
+                var sqlColumns = sqlSelectRemoved.Substring(0, pos);
+                var sqlFrom = sqlSelectRemoved.Substring(pos);
+
+                sqlPage = string.Format("SELECT * FROM (SELECT {0}, ROW_NUMBER() OVER ({1}) peta_rn {2}) peta_paged WHERE peta_rn>@{3} AND peta_rn<=@{4}",
+                                        sqlColumns, sqlOrderBy ?? "ORDER BY (SELECT NULL)", sqlFrom, args.Length, args.Length + 1);
                 args = args.Concat(new object[] { skip, skip + take }).ToArray();
             }
             else if (databaseType == DBType.SqlServerCE)
@@ -774,7 +787,7 @@ namespace Umbraco.Core.Persistence
 				throw new Exception("Unable to parse SQL statement for paged query");
 			if (_dbType == DBType.Oracle && sqlSelectRemoved.StartsWith("*"))
                 throw new Exception("Query must alias '*' when performing a paged query.\neg. select t.* from table t order by t.id");
-            
+
 		    BuildSqlDbSpecificPagingQuery(_dbType, skip, take, sql, sqlSelectRemoved, sqlOrderBy, ref args, out sqlPage);
 		}
 

src/Umbraco.Core/Persistence/PetaPocoExtensions.cs

@@ -3,9 +3,13 @@ using System.Collections.Generic;
 using System.Data;
 using System.Data.Common;
 using System.Data.SqlClient;
+using System.Data.SqlServerCe;
 using System.Linq;
 using System.Text.RegularExpressions;
+using MySql.Data.MySqlClient;
+using StackExchange.Profiling.Data;
 using Umbraco.Core.Logging;
+using Umbraco.Core.Persistence.DatabaseModelDefinitions;
 using Umbraco.Core.Persistence.Querying;
 using Umbraco.Core.Persistence.SqlSyntax;
 
@@ -56,7 +60,7 @@ namespace Umbraco.Core.Persistence
         /// Safely inserts a record, or updates if it exists, based on a unique constraint.
         /// </summary>
         /// <param name="db"></param>
-        /// <param name="poco"></param>       
+        /// <param name="poco"></param>
         /// <returns>The action that executed, either an insert or an update. If an insert occurred and a PK value got generated, the poco object
         /// passed in will contain the updated value.</returns>
         /// <remarks>
@@ -78,7 +82,7 @@ namespace Umbraco.Core.Persistence
         /// </summary>
         /// <param name="db"></param>
         /// <param name="poco"></param>
-        /// <param name="updateArgs"></param>      
+        /// <param name="updateArgs"></param>
         /// <param name="updateCommand">If the entity has a composite key they you need to specify the update command explicitly</param>
         /// <returns>The action that executed, either an insert or an update. If an insert occurred and a PK value got generated, the poco object
         /// passed in will contain the updated value.</returns>
@@ -130,7 +134,7 @@ namespace Umbraco.Core.Persistence
                     if (rowCount > 0)
                         return RecordPersistenceType.Update;
 
-                    // failed: does not exist (due to race cond RC2), need to insert 
+                    // failed: does not exist (due to race cond RC2), need to insert
                     // loop
                 }
             }
@@ -150,7 +154,7 @@ namespace Umbraco.Core.Persistence
             {
                 //this fancy regex will only match a single @ not a double, etc...
                 var regex = new Regex("(?<!@)@(?!@)");
-                return regex.Replace(value, "@@");    
+                return regex.Replace(value, "@@");
             }
             return value;
 
@@ -172,6 +176,13 @@ namespace Umbraco.Core.Persistence
             creator.CreateTable<T>(overwrite);
         }
 
+        /// <summary>
+        /// Performs the bulk insertion in the context of a current transaction with an optional parameter to complete the transaction
+        /// when finished
+        /// </summary>
+        /// <typeparam name="T"></typeparam>
+        /// <param name="db"></param>
+        /// <param name="collection"></param>
         public static void BulkInsertRecords<T>(this Database db, IEnumerable<T> collection)
         {
             //don't do anything if there are no records.
@@ -180,7 +191,7 @@ namespace Umbraco.Core.Persistence
 
             using (var tr = db.GetTransaction())
             {
-                db.BulkInsertRecords(collection, tr, true);
+                db.BulkInsertRecords(collection, tr, SqlSyntaxContext.SqlSyntaxProvider, true, true); // use native, commit
             }
         }
 
@@ -192,55 +203,98 @@ namespace Umbraco.Core.Persistence
         /// <param name="db"></param>
         /// <param name="collection"></param>
         /// <param name="tr"></param>
+        /// <param name="syntaxProvider"></param>
+        /// <param name="useNativeSqlPlatformBulkInsert">
+        /// If this is false this will try to just generate bulk insert statements instead of using the current SQL platform's bulk
+        /// insert logic. For SQLCE, bulk insert statements do not work so if this is false it will insert one at a time.
+        /// </param>
         /// <param name="commitTrans"></param>
-        public static void BulkInsertRecords<T>(this Database db, IEnumerable<T> collection, Transaction tr, bool commitTrans = false)
+        /// <returns>The number of items inserted</returns>
+        public static int BulkInsertRecords<T>(this Database db,
+            IEnumerable<T> collection,
+            Transaction tr,
+            ISqlSyntaxProvider syntaxProvider,
+            bool useNativeSqlPlatformBulkInsert = true,
+            bool commitTrans = false)
         {
+
             //don't do anything if there are no records.
             if (collection.Any() == false)
-                return;
+            {
+                return 0;
+            }
+
+            var pd = Database.PocoData.ForType(typeof(T));
+            if (pd == null) throw new InvalidOperationException("Could not find PocoData for " + typeof(T));
 
             try
             {
-                //if it is sql ce or it is a sql server version less than 2008, we need to do individual inserts.
-                var sqlServerSyntax = SqlSyntaxContext.SqlSyntaxProvider as SqlServerSyntaxProvider;
+                int processed = 0;
 
-                if ((sqlServerSyntax != null && (int)sqlServerSyntax.GetVersionName(db) < (int)SqlServerVersionName.V2008) 
-                    || SqlSyntaxContext.SqlSyntaxProvider is SqlCeSyntaxProvider)
-                {
-                    //SqlCe doesn't support bulk insert statements!
+                var usedNativeSqlPlatformInserts = useNativeSqlPlatformBulkInsert
+                    && NativeSqlPlatformBulkInsertRecords(db, syntaxProvider, pd, collection, out processed);
 
-                    foreach (var poco in collection)
-                    {
-                        db.Insert(poco);
-                    }
-                }
-                else
+                if (usedNativeSqlPlatformInserts == false)
                 {
-                    string[] sqlStatements;
-                    var cmds = db.GenerateBulkInsertCommand(collection, db.Connection, out sqlStatements);
-                    for (var i = 0; i < sqlStatements.Length; i++)
+                    //if it is sql ce or it is a sql server version less than 2008, we need to do individual inserts.
+                    var sqlServerSyntax = syntaxProvider as SqlServerSyntaxProvider;
+                    if ((sqlServerSyntax != null && (int) sqlServerSyntax.GetVersionName(db) < (int) SqlServerVersionName.V2008)
+                        || syntaxProvider is SqlCeSyntaxProvider)
                     {
-                        using (var cmd = cmds[i])
+                        //SqlCe doesn't support bulk insert statements!
+                        foreach (var poco in collection)
                         {
-                            cmd.CommandText = sqlStatements[i];
-                            cmd.ExecuteNonQuery();
+                            db.Insert(poco);
+                        }
+                    }
+                    else
+                    {
+                        //we'll need to generate insert statements instead
+
+                        string[] sqlStatements;
+                        var cmds = db.GenerateBulkInsertCommand(pd, collection, out sqlStatements);
+                        for (var i = 0; i < sqlStatements.Length; i++)
+                        {
+                            using (var cmd = cmds[i])
+                            {
+                                cmd.CommandText = sqlStatements[i];
+                                cmd.ExecuteNonQuery();
+                                processed++;
+                            }
                         }
                     }
                 }
 
                 if (commitTrans)
                 {
-                    tr.Complete();    
+                    tr.Complete();
                 }
+                return processed;
             }
             catch
             {
                 if (commitTrans)
                 {
-                    tr.Dispose();    
+                    tr.Dispose();
                 }
                 throw;
             }
+
+        }
+
+        /// <summary>
+        /// Performs the bulk insertion in the context of a current transaction with an optional parameter to complete the transaction
+        /// when finished
+        /// </summary>
+        /// <typeparam name="T"></typeparam>
+        /// <param name="db"></param>
+        /// <param name="collection"></param>
+        /// <param name="tr"></param>
+        /// <param name="commitTrans"></param>
+        [Obsolete("Use the method that specifies an SqlSyntaxContext instance instead")]
+        public static void BulkInsertRecords<T>(this Database db, IEnumerable<T> collection, Transaction tr, bool commitTrans = false)
+        {
+            db.BulkInsertRecords<T>(collection, tr, SqlSyntaxContext.SqlSyntaxProvider, commitTrans);
         }
 
         /// <summary>
@@ -249,45 +303,36 @@ namespace Umbraco.Core.Persistence
         /// <typeparam name="T"></typeparam>
         /// <param name="db"></param>
         /// <param name="collection"></param>
-        /// <param name="connection"></param>        
         /// <param name="sql"></param>
+        /// <param name="pd"></param>
         /// <returns>Sql commands with populated command parameters required to execute the sql statement</returns>
         /// <remarks>
-        /// The limits for number of parameters are 2100 (in sql server, I think there's many more allowed in mysql). So 
-        /// we need to detect that many params and split somehow. 
-        /// For some reason the 2100 limit is not actually allowed even though the exception from sql server mentions 2100 as a max, perhaps it is 2099 
+        /// The limits for number of parameters are 2100 (in sql server, I think there's many more allowed in mysql). So
+        /// we need to detect that many params and split somehow.
+        /// For some reason the 2100 limit is not actually allowed even though the exception from sql server mentions 2100 as a max, perhaps it is 2099
         /// that is max. I've reduced it to 2000 anyways.
         /// </remarks>
         internal static IDbCommand[] GenerateBulkInsertCommand<T>(
-            this Database db, 
-            IEnumerable<T> collection, 
-            IDbConnection connection,             
+            this Database db,
+            Database.PocoData pd,
+            IEnumerable<T> collection,
             out string[] sql)
         {
-            //A filter used below a few times to get all columns except result cols and not the primary key if it is auto-incremental
-            Func<Database.PocoData, KeyValuePair<string, Database.PocoColumn>, bool> includeColumn = (data, column) =>
-                {
-                    if (column.Value.ResultColumn) return false;
-                    if (data.TableInfo.AutoIncrement && column.Key == data.TableInfo.PrimaryKey) return false;
-                    return true;
-                };
-
-            var pd = Database.PocoData.ForType(typeof(T));
             var tableName = db.EscapeTableName(pd.TableInfo.TableName);
 
             //get all columns to include and format for sql
-            var cols = string.Join(", ", 
+            var cols = string.Join(", ",
                 pd.Columns
-                .Where(c => includeColumn(pd, c))
+                .Where(c => IncludeColumn(pd, c))
                 .Select(c => tableName + "." + db.EscapeSqlIdentifier(c.Key)).ToArray());
 
             var itemArray = collection.ToArray();
 
             //calculate number of parameters per item
-            var paramsPerItem = pd.Columns.Count(i => includeColumn(pd, i));
-            
+            var paramsPerItem = pd.Columns.Count(i => IncludeColumn(pd, i));
+
             //Example calc:
-            // Given: we have 4168 items in the itemArray, each item contains 8 command parameters (values to be inserterted)                
+            // Given: we have 4168 items in the itemArray, each item contains 8 command parameters (values to be inserterted)
             // 2100 / 8 = 262.5
             // Math.Floor(2100 / 8) = 262 items per trans
             // 4168 / 262 = 15.908... = there will be 16 trans in total
@@ -306,14 +351,14 @@ namespace Umbraco.Core.Persistence
                     .Skip(tIndex * (int)itemsPerTrans)
                     .Take((int)itemsPerTrans);
 
-                var cmd = db.CreateCommand(connection, "");
+                var cmd = db.CreateCommand(db.Connection, string.Empty);
                 var pocoValues = new List<string>();
                 var index = 0;
                 foreach (var poco in itemsForTrans)
                 {
                     var values = new List<string>();
                     //get all columns except result cols and not the primary key if it is auto-incremental
-                    foreach (var i in pd.Columns.Where(x => includeColumn(pd, x)))
+                    foreach (var i in pd.Columns.Where(x => IncludeColumn(pd, x)))
                     {
                         db.AddParam(cmd, i.Value.GetValue(poco), "@");
                         values.Add(string.Format("{0}{1}", "@", index++));
@@ -321,14 +366,211 @@ namespace Umbraco.Core.Persistence
                     pocoValues.Add("(" + string.Join(",", values.ToArray()) + ")");
                 }
 
-                var sqlResult = string.Format("INSERT INTO {0} ({1}) VALUES {2}", tableName, cols, string.Join(", ", pocoValues)); 
+                var sqlResult = string.Format("INSERT INTO {0} ({1}) VALUES {2}", tableName, cols, string.Join(", ", pocoValues));
                 sqlQueries.Add(sqlResult);
                 commands.Add(cmd);
             }
 
             sql = sqlQueries.ToArray();
 
-            return commands.ToArray();    
+            return commands.ToArray();
+        }
+
+        /// <summary>
+        /// A filter used below a few times to get all columns except result cols and not the primary key if it is auto-incremental
+        /// </summary>
+        /// <param name="data"></param>
+        /// <param name="column"></param>
+        /// <returns></returns>
+        private static bool IncludeColumn(Database.PocoData data, KeyValuePair<string, Database.PocoColumn> column)
+        {
+            if (column.Value.ResultColumn) return false;
+            if (data.TableInfo.AutoIncrement && column.Key == data.TableInfo.PrimaryKey) return false;
+            return true;
+        }
+
+        /// <summary>
+        /// Bulk insert records with Sql BulkCopy or TableDirect or whatever sql platform specific bulk insert records should be used
+        /// </summary>
+        /// <param name="db"></param>
+        /// <param name="syntaxProvider"></param>
+        /// <param name="pd"></param>
+        /// <param name="collection"></param>
+        /// <param name="processed">The number of records inserted</param>
+        private static bool NativeSqlPlatformBulkInsertRecords<T>(Database db, ISqlSyntaxProvider syntaxProvider, Database.PocoData pd, IEnumerable<T> collection, out int processed)
+        {
+
+            var dbConnection = db.Connection;
+
+            //unwrap the profiled connection if there is one
+            var profiledConnection = dbConnection as ProfiledDbConnection;
+            if (profiledConnection != null)
+            {
+                dbConnection = profiledConnection.InnerConnection;
+            }
+
+            //check if it's SQL or SqlCe
+
+            var sqlConnection = dbConnection as SqlConnection;
+            if (sqlConnection != null)
+            {
+                processed = BulkInsertRecordsSqlServer(db, (SqlServerSyntaxProvider)syntaxProvider, pd, collection);
+                return true;
+            }
+
+            var sqlCeConnection = dbConnection as SqlCeConnection;
+            if (sqlCeConnection != null)
+            {
+                processed = BulkInsertRecordsSqlCe(db, pd, collection);
+                return true;
+            }
+
+            //could not use the SQL server's specific bulk insert operations
+            processed = 0;
+            return false;
+
+        }
+
+        /// <summary>
+        /// Logic used to perform bulk inserts with SqlCe's TableDirect
+        /// </summary>
+        /// <typeparam name="T"></typeparam>
+        /// <param name="db"></param>
+        /// <param name="pd"></param>
+        /// <param name="collection"></param>
+        /// <returns></returns>
+        internal static int BulkInsertRecordsSqlCe<T>(Database db,
+            Database.PocoData pd,
+            IEnumerable<T> collection)
+        {
+            var cols = pd.Columns.ToArray();
+
+            using (var cmd = db.CreateCommand(db.Connection, string.Empty))
+            {
+                cmd.CommandText = pd.TableInfo.TableName;
+                cmd.CommandType = CommandType.TableDirect;
+                //cmd.Transaction = GetTypedTransaction<SqlCeTransaction>(db.Connection.);
+
+                //get the real command
+                using (var sqlCeCommand = GetTypedCommand<SqlCeCommand>(cmd))
+                {
+                    // This seems to cause problems, I think this is primarily used for retrieval, not
+                    // inserting. see: https://msdn.microsoft.com/en-us/library/system.data.sqlserverce.sqlcecommand.indexname%28v=vs.100%29.aspx?f=255&MSPPError=-2147217396
+                    //sqlCeCommand.IndexName = pd.TableInfo.PrimaryKey;
+
+                    var count = 0;
+                    using (var rs = sqlCeCommand.ExecuteResultSet(ResultSetOptions.Updatable))
+                    {
+                        var rec = rs.CreateRecord();
+
+                        foreach (var item in collection)
+                        {
+                            for (var i = 0; i < cols.Length; i++)
+                            {
+                                //skip the index if this shouldn't be included (i.e. PK)
+                                if (IncludeColumn(pd, cols[i]))
+                                {
+                                    var val = cols[i].Value.GetValue(item);
+                                    rec.SetValue(i, val);
+                                }
+                            }
+                            rs.Insert(rec);
+                            count++;
+                        }
+                    }
+                    return count;
+                }
+
+            }
+        }
+
+        /// <summary>
+        /// Logic used to perform bulk inserts with SqlServer's BulkCopy
+        /// </summary>
+        /// <typeparam name="T"></typeparam>
+        /// <param name="db"></param>
+        /// <param name="sqlSyntaxProvider"></param>
+        /// <param name="pd"></param>
+        /// <param name="collection"></param>
+        /// <returns></returns>
+        internal static int BulkInsertRecordsSqlServer<T>(Database db, SqlServerSyntaxProvider sqlSyntaxProvider,
+            Database.PocoData pd, IEnumerable<T> collection)
+        {
+            //NOTE: We need to use the original db.Connection here to create the command, but we need to pass in the typed
+            // connection below to the SqlBulkCopy
+            using (var cmd = db.CreateCommand(db.Connection, string.Empty))
+            {
+                using (var copy = new SqlBulkCopy(
+                    GetTypedConnection<SqlConnection>(db.Connection),
+                    SqlBulkCopyOptions.Default,
+                    GetTypedTransaction<SqlTransaction>(cmd.Transaction))
+                {
+                    BulkCopyTimeout = 10000,
+                    DestinationTableName = pd.TableInfo.TableName
+                })
+                {
+                    //var cols = pd.Columns.Where(x => IncludeColumn(pd, x)).Select(x => x.Value).ToArray();
+
+                    using (var bulkReader = new PocoDataDataReader<T, SqlServerSyntaxProvider>(collection, pd, sqlSyntaxProvider))
+                    {
+                        copy.WriteToServer(bulkReader);
+
+                        return bulkReader.RecordsAffected;
+                    }
+                }
+            }
+        }
+
+
+        /// <summary>
+        /// Returns the underlying connection as a typed connection - this is used to unwrap the profiled mini profiler stuff
+        /// </summary>
+        /// <typeparam name="TConnection"></typeparam>
+        /// <param name="connection"></param>
+        /// <returns></returns>
+        private static TConnection GetTypedConnection<TConnection>(IDbConnection connection)
+            where TConnection : class, IDbConnection
+        {
+            var profiled = connection as ProfiledDbConnection;
+            if (profiled != null)
+            {
+                return profiled.InnerConnection as TConnection;
+            }
+            return connection as TConnection;
+        }
+
+        /// <summary>
+        /// Returns the underlying connection as a typed connection - this is used to unwrap the profiled mini profiler stuff
+        /// </summary>
+        /// <typeparam name="TTransaction"></typeparam>
+        /// <param name="connection"></param>
+        /// <returns></returns>
+        private static TTransaction GetTypedTransaction<TTransaction>(IDbTransaction connection)
+            where TTransaction : class, IDbTransaction
+        {
+            var profiled = connection as ProfiledDbTransaction;
+            if (profiled != null)
+            {
+                return profiled.WrappedTransaction as TTransaction;
+            }
+            return connection as TTransaction;
+        }
+
+        /// <summary>
+        /// Returns the underlying connection as a typed connection - this is used to unwrap the profiled mini profiler stuff
+        /// </summary>
+        /// <typeparam name="TCommand"></typeparam>
+        /// <param name="command"></param>
+        /// <returns></returns>
+        private static TCommand GetTypedCommand<TCommand>(IDbCommand command)
+            where TCommand : class, IDbCommand
+        {
+            var profiled = command as ProfiledDbCommand;
+            if (profiled != null)
+            {
+                return profiled.InternalCommand as TCommand;
+            }
+            return command as TCommand;
         }
 
         [Obsolete("Use the DatabaseSchemaHelper instead")]
@@ -405,8 +647,8 @@ namespace Umbraco.Core.Persistence
             return ApplicationContext.Current.DatabaseContext.DatabaseProvider;
         }
 
-        
+
     }
 
-    
+
 }

src/Umbraco.Core/Persistence/PetaPocoSqlExtensions.cs

@@ -27,9 +27,17 @@ namespace Umbraco.Core.Persistence
             return sql.From(sqlSyntax.GetQuotedTableName(tableName));
         }
 
+        [Obsolete("Use the overload specifying ISqlSyntaxProvider instead")]
         public static Sql Where<T>(this Sql sql, Expression<Func<T, bool>> predicate)
         {
-            var expresionist = new PocoToSqlExpressionHelper<T>();
+            var expresionist = new PocoToSqlExpressionVisitor<T>();
+            var whereExpression = expresionist.Visit(predicate);
+            return sql.Where(whereExpression, expresionist.GetSqlParameters());
+        }
+
+        public static Sql Where<T>(this Sql sql, Expression<Func<T, bool>> predicate, ISqlSyntaxProvider sqlSyntax)
+        {
+            var expresionist = new PocoToSqlExpressionVisitor<T>(sqlSyntax);
             var whereExpression = expresionist.Visit(predicate);
             return sql.Where(whereExpression, expresionist.GetSqlParameters());
         }

src/Umbraco.Core/Persistence/PocoDataDataReader.cs

@@ -0,0 +1,159 @@
+using System;
+using System.Collections.Generic;
+using System.Data;
+using System.Linq;
+using Umbraco.Core.Persistence.DatabaseAnnotations;
+using Umbraco.Core.Persistence.DatabaseModelDefinitions;
+using Umbraco.Core.Persistence.SqlSyntax;
+
+namespace Umbraco.Core.Persistence
+{
+    /// <summary>
+    /// A data reader used for reading collections of PocoData entity types
+    /// </summary>
+    /// <remarks>
+    /// We are using a custom data reader so that tons of memory is not consumed when rebuilding this table, previously
+    /// we'd generate SQL insert statements, but we'd have to put all of the XML structures into memory first. Alternatively
+    /// we can use .net's DataTable, but this also requires putting everything into memory. By using a DataReader we don't have to 
+    /// store every content item and it's XML structure in memory to get it into the DB, we can stream it into the db with this 
+    /// reader.
+    /// </remarks>
+    internal class PocoDataDataReader<T, TSyntax> : BulkDataReader 
+        where TSyntax : ISqlSyntaxProvider
+    {        
+        private readonly MicrosoftSqlSyntaxProviderBase<TSyntax> _sqlSyntaxProvider;
+        private readonly TableDefinition _tableDefinition;
+        private readonly Database.PocoColumn[] _readerColumns;
+        private readonly IEnumerator<T> _enumerator;
+        private readonly ColumnDefinition[] _columnDefinitions;
+        private int _recordsAffected = -1;
+
+        public PocoDataDataReader(
+            IEnumerable<T> dataSource,             
+            Database.PocoData pd,
+            MicrosoftSqlSyntaxProviderBase<TSyntax> sqlSyntaxProvider)
+        {
+            if (dataSource == null) throw new ArgumentNullException("dataSource");            
+            if (sqlSyntaxProvider == null) throw new ArgumentNullException("sqlSyntaxProvider");
+
+            _tableDefinition = DefinitionFactory.GetTableDefinition(sqlSyntaxProvider, pd.type);
+            if (_tableDefinition == null) throw new InvalidOperationException("No table definition found for type " + pd.type);
+            
+            _readerColumns = pd.Columns.Select(x => x.Value).ToArray();
+            _sqlSyntaxProvider = sqlSyntaxProvider;
+            _enumerator = dataSource.GetEnumerator();
+            _columnDefinitions = _tableDefinition.Columns.ToArray();
+            
+        }
+
+        protected override string SchemaName
+        {
+            get { return _tableDefinition.SchemaName; }
+        }
+
+        protected override string TableName
+        {
+            get { return _tableDefinition.Name; }
+        }
+
+        public override int RecordsAffected
+        {
+            get { return _recordsAffected <= 0 ? -1 : _recordsAffected; }
+        }
+
+        /// <summary>
+        /// This will automatically add the schema rows based on the Poco table definition and the columns passed in
+        /// </summary>
+        protected override void AddSchemaTableRows()
+        {
+            //var colNames = _readerColumns.Select(x => x.ColumnName).ToArray();
+            //foreach (var col in _columnDefinitions.Where(x => colNames.Contains(x.Name, StringComparer.OrdinalIgnoreCase)))
+            foreach (var col in _columnDefinitions)
+            {
+                var sqlDbType = SqlDbType.NVarChar;
+                if (col.HasSpecialDbType)
+                {
+                    //get the SqlDbType from the 'special type'
+                    switch (col.DbType)
+                    {
+                        case SpecialDbTypes.NTEXT:
+                            sqlDbType = SqlDbType.NText;
+                            break;
+                        case SpecialDbTypes.NCHAR:
+                            sqlDbType = SqlDbType.NChar;
+                            break;
+                        default:
+                            throw new ArgumentOutOfRangeException();
+                    }
+                }
+                else if (col.Type.HasValue)
+                {
+                    //get the SqlDbType from the DbType
+                    sqlDbType = _sqlSyntaxProvider.GetSqlDbType(col.Type.Value);
+                }
+                else
+                {
+                    //get the SqlDbType from the clr type
+                    sqlDbType = _sqlSyntaxProvider.GetSqlDbType(col.PropertyType);
+                }
+                
+                AddSchemaTableRow(
+                    col.Name, 
+                    col.Size > 0 ? (int?)col.Size : null,
+                    col.Precision > 0  ? (short?)col.Precision : null, 
+                    null, col.IsUnique, col.IsIdentity, col.IsNullable, sqlDbType, 
+                    null, null, null, null, null);                
+            }
+
+        }
+
+        /// <summary>
+        /// Get the value from the column index for the current object
+        /// </summary>
+        /// <param name="i"></param>
+        /// <returns></returns>
+        public override object GetValue(int i)
+        {
+            if (_enumerator.Current != null)
+            {
+                return _readerColumns[i].GetValue(_enumerator.Current);
+                //return _columnDefinitions[i]. .GetValue(_enumerator.Current);
+            }
+
+            return null;
+            //TODO: Or throw ?
+        }
+
+        /// <summary>
+        /// Advance the cursor
+        /// </summary>
+        /// <returns></returns>
+        public override bool Read()
+        {
+            var result = _enumerator.MoveNext();
+            if (result)
+            {
+                if (_recordsAffected == -1)
+                {
+                    _recordsAffected = 0;
+                }
+                _recordsAffected++;
+            }
+            return result;
+        }
+
+        /// <summary>
+        /// Ensure the enumerator is disposed
+        /// </summary>
+        /// <param name="disposing"></param>
+        protected override void Dispose(bool disposing)
+        {
+            base.Dispose(disposing);
+
+            if (disposing)
+            {
+                _enumerator.Dispose();
+            }
+        }
+    }
+}

src/Umbraco.Core/Persistence/Querying/IQuery.cs

@@ -1,31 +1,8 @@
 using System;
-using System.Collections.Generic;
 using System.Linq.Expressions;
 
 namespace Umbraco.Core.Persistence.Querying
 {
-    /// <summary>
-    /// SD: This is a horrible hack but unless we break compatibility with anyone who's actually implemented IQuery{T} there's not much we can do.
-    /// The IQuery{T} interface is useless without having a GetWhereClauses method and cannot be used for tests.
-    /// We have to wait till v8 to make this change I suppose.
-    /// </summary>
-    internal static class QueryExtensions
-    {
-        /// <summary>
-        /// Returns all translated where clauses and their sql parameters
-        /// </summary>
-        /// <returns></returns>
-        public static IEnumerable<Tuple<string, object[]>> GetWhereClauses<T>(this IQuery<T> query)
-        {
-            var q = query as Query<T>;
-            if (q == null)
-            {
-                throw new NotSupportedException(typeof(IQuery<T>) + " cannot be cast to " + typeof(Query<T>));
-            }
-            return q.GetWhereClauses();
-        }
-    }
-
     /// <summary>
     /// Represents a query for building Linq translatable SQL queries
     /// </summary>

src/Umbraco.Core/Persistence/Querying/ModelToSqlExpressionHelper.cs

@@ -1,62 +0,0 @@
-using System;
-using System.Collections.Generic;
-using System.Collections.ObjectModel;
-using System.Diagnostics;
-using System.Globalization;
-using System.Linq;
-using System.Linq.Expressions;
-using System.Text;
-using Umbraco.Core.Persistence.Mappers;
-using Umbraco.Core.Persistence.SqlSyntax;
-
-namespace Umbraco.Core.Persistence.Querying
-{
-    internal class ModelToSqlExpressionHelper<T> : BaseExpressionHelper<T>
-    {
-
-        private readonly BaseMapper _mapper;
-
-        public ModelToSqlExpressionHelper()
-        {
-            _mapper = MappingResolver.Current.ResolveMapperByType(typeof(T));
-        }
-        
-        protected override string VisitMemberAccess(MemberExpression m)
-        {
-            if (m.Expression != null && 
-                m.Expression.NodeType == ExpressionType.Parameter 
-                && m.Expression.Type == typeof(T))
-            {
-                var field = _mapper.Map(m.Member.Name, true);
-                if (field.IsNullOrWhiteSpace()) 
-                    throw new InvalidOperationException("The mapper returned an empty field for the member name: " + m.Member.Name);
-                return field;                
-            }
-
-            if (m.Expression != null && m.Expression.NodeType == ExpressionType.Convert)
-            {
-                var field = _mapper.Map(m.Member.Name, true);
-                if (field.IsNullOrWhiteSpace()) 
-                    throw new InvalidOperationException("The mapper returned an empty field for the member name: " + m.Member.Name);
-                return field;
-            }
-            
-            var member = Expression.Convert(m, typeof(object));
-            var lambda = Expression.Lambda<Func<object>>(member);
-            var getter = lambda.Compile();
-            object o = getter();
-
-            SqlParameters.Add(o);
-            return string.Format("@{0}", SqlParameters.Count - 1);
-
-            //return GetQuotedValue(o, o != null ? o.GetType() : null);
-
-        }
-    
-        //protected bool IsFieldName(string quotedExp)
-        //{
-        //    //Not entirely sure this is reliable, but its better then simply returning true
-        //    return quotedExp.LastIndexOf("'", StringComparison.InvariantCultureIgnoreCase) + 1 != quotedExp.Length;
-        //}
-    }
-}

src/Umbraco.Core/Persistence/Querying/ModelToSqlExpressionVisitor.cs

@@ -0,0 +1,74 @@
+using System;
+using System.Linq.Expressions;
+using Umbraco.Core.Persistence.Mappers;
+using Umbraco.Core.Persistence.SqlSyntax;
+
+namespace Umbraco.Core.Persistence.Querying
+{
+    /// <summary>
+    /// An expression tree parser to create SQL statements and SQL parameters based on a strongly typed expression,
+    /// based on Umbraco's business logic models.
+    /// </summary>
+    /// <remarks>This object is stateful and cannot be re-used to parse an expression.</remarks>
+    internal class ModelToSqlExpressionVisitor<T> : ExpressionVisitorBase
+    {
+        private readonly BaseMapper _mapper;
+
+        public ModelToSqlExpressionVisitor(ISqlSyntaxProvider sqlSyntax, BaseMapper mapper)
+            : base(sqlSyntax)
+        {
+            _mapper = mapper;
+        }
+
+        public ModelToSqlExpressionVisitor()
+            : this(SqlSyntaxContext.SqlSyntaxProvider, MappingResolver.Current.ResolveMapperByType(typeof(T)))
+        { }
+
+        protected override string VisitMemberAccess(MemberExpression m)
+        {
+            if (m.Expression != null &&
+                m.Expression.NodeType == ExpressionType.Parameter
+                && m.Expression.Type == typeof(T))
+            {
+                //don't execute if compiled
+                if (Visited == false)
+                {
+                    var field = _mapper.Map(m.Member.Name, true);
+                    if (field.IsNullOrWhiteSpace())
+                        throw new InvalidOperationException("The mapper returned an empty field for the member name: " + m.Member.Name);
+                    return field;
+                }
+                //already compiled, return
+                return string.Empty;
+            }
+
+            if (m.Expression != null && m.Expression.NodeType == ExpressionType.Convert)
+            {
+                //don't execute if compiled
+                if (Visited == false)
+                {
+                    var field = _mapper.Map(m.Member.Name, true);
+                    if (field.IsNullOrWhiteSpace())
+                        throw new InvalidOperationException("The mapper returned an empty field for the member name: " + m.Member.Name);
+                    return field;
+                }
+                //already compiled, return
+                return string.Empty;
+            }
+
+            var member = Expression.Convert(m, typeof(object));
+            var lambda = Expression.Lambda<Func<object>>(member);
+            var getter = lambda.Compile();
+            var o = getter();
+
+            SqlParameters.Add(o);
+
+            //don't execute if compiled
+            if (Visited == false)
+                return string.Format("@{0}", SqlParameters.Count - 1);
+            //already compiled, return
+            return string.Empty;
+
+        }
+    }
+}

src/Umbraco.Core/Persistence/Querying/PocoToSqlExpressionHelper.cs

@@ -1,62 +0,0 @@
-using System;
-using System.Collections;
-using System.Collections.Generic;
-using System.Collections.ObjectModel;
-using System.Linq;
-using System.Linq.Expressions;
-using System.Text;
-using Umbraco.Core.Persistence.SqlSyntax;
-
-namespace Umbraco.Core.Persistence.Querying
-{
-    internal class PocoToSqlExpressionHelper<T> : BaseExpressionHelper<T>
-    {
-        private readonly Database.PocoData _pd;
-
-        public PocoToSqlExpressionHelper()
-        {
-            _pd = new Database.PocoData(typeof(T));
-        }
-        
-        protected override string VisitMemberAccess(MemberExpression m)
-        {
-            if (m.Expression != null &&
-               m.Expression.NodeType == ExpressionType.Parameter
-               && m.Expression.Type == typeof(T))
-            {
-                string field = GetFieldName(_pd, m.Member.Name);
-                return field;
-            }
-
-            if (m.Expression != null && m.Expression.NodeType == ExpressionType.Convert)
-            {
-                string field = GetFieldName(_pd, m.Member.Name);
-                return field;
-            }
-
-            var member = Expression.Convert(m, typeof(object));
-            var lambda = Expression.Lambda<Func<object>>(member);
-            var getter = lambda.Compile();
-            object o = getter();
-
-            SqlParameters.Add(o);
-            return string.Format("@{0}", SqlParameters.Count - 1);
-
-            //return GetQuotedValue(o, o != null ? o.GetType() : null);
-
-        }
-        
-        protected virtual string GetFieldName(Database.PocoData pocoData, string name)
-        {
-            var column = pocoData.Columns.FirstOrDefault(x => x.Value.PropertyInfo.Name == name);
-            return string.Format("{0}.{1}",
-                SqlSyntaxContext.SqlSyntaxProvider.GetQuotedTableName(pocoData.TableInfo.TableName),
-                SqlSyntaxContext.SqlSyntaxProvider.GetQuotedColumnName(column.Value.ColumnName));
-        }
-        
-        //protected bool IsFieldName(string quotedExp)
-        //{
-        //    return true;
-        //}
-    }
-}

src/Umbraco.Core/Persistence/Querying/PocoToSqlExpressionVisitor.cs

@@ -0,0 +1,81 @@
+using System;
+using System.Linq;
+using System.Linq.Expressions;
+using Umbraco.Core.Persistence.SqlSyntax;
+
+namespace Umbraco.Core.Persistence.Querying
+{
+    /// <summary>
+    /// An expression tree parser to create SQL statements and SQL parameters based on a strongly typed expression,
+    /// based on Umbraco's DTOs.
+    /// </summary>
+    /// <remarks>This object is stateful and cannot be re-used to parse an expression.</remarks>
+    internal class PocoToSqlExpressionVisitor<T> : ExpressionVisitorBase
+    {
+        private readonly Database.PocoData _pd;
+
+
+        public PocoToSqlExpressionVisitor(ISqlSyntaxProvider syntaxProvider)
+            : base(syntaxProvider)
+        {
+            
+        }
+
+        [Obsolete("Use the overload the specifies a SqlSyntaxProvider")]
+        public PocoToSqlExpressionVisitor()
+            : base(SqlSyntaxContext.SqlSyntaxProvider)
+        {
+            _pd = new Database.PocoData(typeof(T));
+        }
+
+        protected override string VisitMemberAccess(MemberExpression m)
+        {
+            if (m.Expression != null &&
+               m.Expression.NodeType == ExpressionType.Parameter
+               && m.Expression.Type == typeof(T))
+            {
+                //don't execute if compiled
+                if (Visited == false)
+                {
+                    string field = GetFieldName(_pd, m.Member.Name);
+                    return field;
+                }
+                //already compiled, return
+                return string.Empty;
+            }
+
+            if (m.Expression != null && m.Expression.NodeType == ExpressionType.Convert)
+            {
+                //don't execute if compiled
+                if (Visited == false)
+                {
+                    string field = GetFieldName(_pd, m.Member.Name);
+                    return field;
+                }
+                //already compiled, return
+                return string.Empty;
+            }
+
+            var member = Expression.Convert(m, typeof(object));
+            var lambda = Expression.Lambda<Func<object>>(member);
+            var getter = lambda.Compile();
+            var o = getter();
+
+            SqlParameters.Add(o);
+
+            //don't execute if compiled
+            if (Visited == false)
+                return string.Format("@{0}", SqlParameters.Count - 1);
+            //already compiled, return
+            return string.Empty;
+        }
+
+        protected virtual string GetFieldName(Database.PocoData pocoData, string name)
+        {
+            var column = pocoData.Columns.FirstOrDefault(x => x.Value.PropertyInfo.Name == name);
+            return string.Format("{0}.{1}",
+                SqlSyntax.GetQuotedTableName(pocoData.TableInfo.TableName),
+                SqlSyntax.GetQuotedColumnName(column.Value.ColumnName));
+        }
+    }
+}

src/Umbraco.Core/Persistence/Querying/Query.cs

@@ -30,7 +30,8 @@ namespace Umbraco.Core.Persistence.Querying
         {
             if (predicate != null)
             {
-                var expressionHelper = new ModelToSqlExpressionHelper<T>();
+                //TODO: This should have an SqlSyntax object passed in, this ctor is relying on a singleton
+                var expressionHelper = new ModelToSqlExpressionVisitor<T>();
                 string whereExpression = expressionHelper.Visit(predicate);
 
                 _wheres.Add(new Tuple<string, object[]>(whereExpression, expressionHelper.GetSqlParameters()));

src/Umbraco.Core/Persistence/Querying/QueryExtensions.cs

@@ -0,0 +1,27 @@
+using System;
+using System.Collections.Generic;
+
+namespace Umbraco.Core.Persistence.Querying
+{
+    /// <summary>
+    /// SD: This is a horrible hack but unless we break compatibility with anyone who's actually implemented IQuery{T} there's not much we can do.
+    /// The IQuery{T} interface is useless without having a GetWhereClauses method and cannot be used for tests.
+    /// We have to wait till v8 to make this change I suppose.
+    /// </summary>
+    internal static class QueryExtensions
+    {
+        /// <summary>
+        /// Returns all translated where clauses and their sql parameters
+        /// </summary>
+        /// <returns></returns>
+        public static IEnumerable<Tuple<string, object[]>> GetWhereClauses<T>(this IQuery<T> query)
+        {
+            var q = query as Query<T>;
+            if (q == null)
+            {
+                throw new NotSupportedException(typeof(IQuery<T>) + " cannot be cast to " + typeof(Query<T>));
+            }
+            return q.GetWhereClauses();
+        }
+    }
+}

src/Umbraco.Core/Persistence/Repositories/ContentRepository.cs

@@ -1,22 +1,14 @@
 using System;
 using System.Collections.Generic;
-using System.Data;
 using System.Globalization;
 using System.Linq;
-using System.Linq.Expressions;
-using System.Net.Http.Headers;
-using System.Text;
 using System.Xml;
 using System.Xml.Linq;
-using Umbraco.Core.Configuration;
-using Umbraco.Core.Dynamics;
-using Umbraco.Core.IO;
 using Umbraco.Core.Logging;
 using Umbraco.Core.Models;
 using Umbraco.Core.Models.EntityBase;
 using Umbraco.Core.Models.Membership;
 using Umbraco.Core.Models.Rdbms;
-
 using Umbraco.Core.Persistence.DatabaseModelDefinitions;
 using Umbraco.Core.Persistence.Factories;
 using Umbraco.Core.Persistence.Querying;
@@ -81,7 +73,7 @@ namespace Umbraco.Core.Persistence.Repositories
             var sql = GetBaseQuery(false);
             if (ids.Any())
             {
-                sql.Where("umbracoNode.id in (@ids)", new { ids = ids });
+                sql.Where("umbracoNode.id in (@ids)", new { ids });
             }
 
             //we only want the newest ones with this method
@@ -102,7 +94,7 @@ namespace Umbraco.Core.Persistence.Repositories
             return ProcessQuery(sql);
         }
 
-        #endregion
+        #endregion        
 
         #region Overrides of PetaPocoRepositoryBase<IContent>
 
@@ -173,90 +165,57 @@ namespace Umbraco.Core.Persistence.Repositories
 
         #region Overrides of VersionableRepositoryBase<IContent>
 
-        public void RebuildXmlStructures(Func<IContent, XElement> serializer, int groupSize = 5000, IEnumerable<int> contentTypeIds = null)
+        public void RebuildXmlStructures(Func<IContent, XElement> serializer, int groupSize = 200, IEnumerable<int> contentTypeIds = null)
         {
+            // the previous way of doing this was to run it all in one big transaction,
+            // and to bulk-insert groups of xml rows - which works, until the transaction
+            // times out - and besides, because v7 transactions are ReadCommited, it does
+            // not bring much safety - so this reverts to updating each record individually,
+            // and it may be slower in the end, but should be more resilient.
 
-            //Ok, now we need to remove the data and re-insert it, we'll do this all in one transaction too.
-            using (var tr = Database.GetTransaction())
+            var baseId = 0;
+            var contentTypeIdsA = contentTypeIds == null ? new int[0] : contentTypeIds.ToArray();
+            while (true)
             {
-                //Remove all the data first, if anything fails after this it's no problem the transaction will be reverted
-                if (contentTypeIds == null)
-                {
-                    var subQuery = new Sql()
-                            .Select("DISTINCT cmsContentXml.nodeId")
-                            .From<ContentXmlDto>()
-                            .InnerJoin<DocumentDto>()
-                            .On<ContentXmlDto, DocumentDto>(left => left.NodeId, right => right.NodeId);
+                // get the next group of nodes
+                var query = GetBaseQuery(false);
+                if (contentTypeIdsA.Length > 0)
+                    query = query
+                        .WhereIn<ContentDto>(x => x.ContentTypeId, contentTypeIdsA, SqlSyntax);
+                query = query
+                    .Where<NodeDto>(x => x.NodeId > baseId && x.Trashed == false)
+                    .Where<DocumentDto>(x => x.Published)
+                    .OrderBy<NodeDto>(x => x.NodeId, SqlSyntax);
+                var xmlItems = ProcessQuery(SqlSyntax.SelectTop(query, groupSize))
+                    .Select(x => new ContentXmlDto { NodeId = x.Id, Xml = serializer(x).ToString() })
+                    .ToList();
 
-                    var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                    Database.Execute(deleteSql);
-                }
-                else
+                // no more nodes, break
+                if (xmlItems.Count == 0) break;
+
+                foreach (var xmlItem in xmlItems)
                 {
-                    foreach (var id in contentTypeIds)
+                    try
                     {
-                        var id1 = id;
-                        var subQuery = new Sql()
-                            .Select("cmsDocument.nodeId")
-                            .From<DocumentDto>()
-                            .InnerJoin<ContentDto>()
-                            .On<DocumentDto, ContentDto>(left => left.NodeId, right => right.NodeId)
-                            .Where<DocumentDto>(dto => dto.Published)
-                            .Where<ContentDto>(dto => dto.ContentTypeId == id1);
-
-                        var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                        Database.Execute(deleteSql);
+                        // should happen in most cases, then it tries to insert, and it should work
+                        // unless the node has been deleted, and we just report the exception
+                        Database.InsertOrUpdate(xmlItem);
+                    }
+                    catch (Exception e)
+                    {
+                        Logger.Error<MediaRepository>("Could not rebuild XML for nodeId=" + xmlItem.NodeId, e);
                     }
                 }
-
-                //now insert the data, again if something fails here, the whole transaction is reversed
-                if (contentTypeIds == null)
-                {
-                    var query = Query<IContent>.Builder.Where(x => x.Published == true);
-                    RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                }
-                else
-                {
-                    foreach (var contentTypeId in contentTypeIds)
-                    {
-                        //copy local
-                        var id = contentTypeId;
-                        var query = Query<IContent>.Builder.Where(x => x.Published == true && x.ContentTypeId == id && x.Trashed == false);
-                        RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                    }
-                }
-
-                tr.Complete();
+                baseId = xmlItems.Last().NodeId;
             }
         }
 
-        private void RebuildXmlStructuresProcessQuery(Func<IContent, XElement> serializer, IQuery<IContent> query, Transaction tr, int pageSize)
+        public override IEnumerable<IContent> GetAllVersions(int id)
         {
-            var pageIndex = 0;
-            var total = long.MinValue;
-            var processed = 0;
-            do
-            {
-                //NOTE: This is an important call, we cannot simply make a call to:
-                //  GetPagedResultsByQuery(query, pageIndex, pageSize, out total, "Path", Direction.Ascending);
-                // because that method is used to query 'latest' content items where in this case we don't necessarily
-                // want latest content items because a pulished content item might not actually be the latest.
-                // see: http://issues.umbraco.org/issue/U4-6322 & http://issues.umbraco.org/issue/U4-5982
-                var descendants = GetPagedResultsByQuery<DocumentDto, Content>(query, pageIndex, pageSize, out total,
-                    new Tuple<string, string>("cmsDocument", "nodeId"),
-                    ProcessQuery, "Path", Direction.Ascending, true);
-
-                var xmlItems = (from descendant in descendants
-                                let xml = serializer(descendant)
-                                select new ContentXmlDto { NodeId = descendant.Id, Xml = xml.ToDataString() }).ToArray();
-
-                //bulk insert it into the database
-                Database.BulkInsertRecords(xmlItems, tr);
-
-                processed += xmlItems.Length;
-
-                pageIndex++;
-            } while (processed < total);
+            var sql = GetBaseQuery(false)
+                .Where(GetBaseWhereClause(), new { Id = id })
+                .OrderByDescending<ContentVersionDto>(x => x.VersionDate, SqlSyntax);
+            return ProcessQuery(sql, true);
         }
 
         public override IContent GetByVersion(Guid versionId)
@@ -383,6 +342,7 @@ namespace Umbraco.Core.Persistence.Repositories
 
             //Update with new correct path
             nodeDto.Path = string.Concat(parent.Path, ",", nodeDto.NodeId);
+            nodeDto.ValidatePathWithException();
             Database.Update(nodeDto);
 
             //Update entity with correct values
@@ -520,6 +480,7 @@ namespace Umbraco.Core.Persistence.Repositories
 
             //Updates the (base) node data - umbracoNode
             var nodeDto = dto.ContentVersionDto.ContentDto.NodeDto;
+            nodeDto.ValidatePathWithException();
             var o = Database.Update(nodeDto);
 
             //Only update this DTO if the contentType has actually changed
@@ -667,29 +628,9 @@ namespace Umbraco.Core.Persistence.Repositories
                                 .OrderBy<NodeDto>(x => x.Level, SqlSyntax)
                                 .OrderBy<NodeDto>(x => x.SortOrder, SqlSyntax);
 
-            //NOTE: This doesn't allow properties to be part of the query
-            var dtos = Database.Fetch<DocumentDto, ContentVersionDto, ContentDto, NodeDto, DocumentPublishedReadOnlyDto>(sql);
-
-            foreach (var dto in dtos)
-            {
-                //Check in the cache first. If it exists there AND it is published
-                // then we can use that entity. Otherwise if it is not published (which can be the case
-                // because we only store the 'latest' entries in the cache which might not be the published
-                // version)
-                var fromCache = RuntimeCache.GetCacheItem<IContent>(GetCacheIdKey<IContent>(dto.NodeId));
-                //var fromCache = TryGetFromCache(dto.NodeId);
-                if (fromCache != null && fromCache.Published)
-                {
-                    yield return fromCache;
-                }
-                else
-                {
-                    yield return CreateContentFromDto(dto, dto.VersionId, sql);
-                }
-            }
+            return ProcessQuery(sql, true);
         }
 
-
         /// <summary>
         /// This builds the Xml document used for the XML cache
         /// </summary>
@@ -863,7 +804,7 @@ order by umbracoNode.level, umbracoNode.parentID, umbracoNode.sortOrder";
 
             return GetPagedResultsByQuery<DocumentDto, Content>(query, pageIndex, pageSize, out totalRecords,
                 new Tuple<string, string>("cmsDocument", "nodeId"),
-                ProcessQuery, orderBy, orderDirection, orderBySystemField,
+                sql => ProcessQuery(sql), orderBy, orderDirection, orderBySystemField,
                 filterCallback);
 
         }
@@ -894,83 +835,80 @@ order by umbracoNode.level, umbracoNode.parentID, umbracoNode.sortOrder";
             return base.GetDatabaseFieldNameForOrderBy(orderBy);
         }
 
-        private IEnumerable<IContent> ProcessQuery(Sql sql)
+        private IEnumerable<IContent> ProcessQuery(Sql sql, bool withCache = false)
         {
-            //NOTE: This doesn't allow properties to be part of the query
+            // fetch returns a list so it's ok to iterate it in this method
             var dtos = Database.Fetch<DocumentDto, ContentVersionDto, ContentDto, NodeDto, DocumentPublishedReadOnlyDto>(sql);
+            if (dtos.Count == 0) return Enumerable.Empty<IContent>();
 
-            //nothing found
-            if (dtos.Any() == false) return Enumerable.Empty<IContent>();
+            var content = new IContent[dtos.Count];
+            var defs = new List<DocumentDefinition>();
+            var templateIds = new List<int>();
 
-            //content types
-            //NOTE: This should be ok for an SQL 'IN' statement, there shouldn't be an insane amount of content types
-            var contentTypes = _contentTypeRepository.GetAll(dtos.Select(x => x.ContentVersionDto.ContentDto.ContentTypeId).ToArray())
-                .ToArray();
-
-
-            var ids = dtos
-                .Where(dto => dto.TemplateId.HasValue && dto.TemplateId.Value > 0)
-                .Select(x => x.TemplateId.Value).ToArray();
-
-            //NOTE: This should be ok for an SQL 'IN' statement, there shouldn't be an insane amount of content types
-            var templates = ids.Length == 0 ? Enumerable.Empty<ITemplate>() : _templateRepository.GetAll(ids).ToArray();
-
-            var dtosWithContentTypes = dtos
-                //This select into and null check are required because we don't have a foreign damn key on the contentType column
-                // http://issues.umbraco.org/issue/U4-5503
-                .Select(x => new { dto = x, contentType = contentTypes.FirstOrDefault(ct => ct.Id == x.ContentVersionDto.ContentDto.ContentTypeId) })
-                .Where(x => x.contentType != null)
-                .ToArray();
-
-            //Go get the property data for each document
-            var docDefs = dtosWithContentTypes.Select(d => new DocumentDefinition(
-                d.dto.NodeId,
-                d.dto.VersionId,
-                d.dto.ContentVersionDto.VersionDate,
-                d.dto.ContentVersionDto.ContentDto.NodeDto.CreateDate,
-                d.contentType));
-
-            var propertyData = GetPropertyCollection(sql, docDefs);
-
-            return dtosWithContentTypes.Select(d => CreateContentFromDto(
-                d.dto,
-                contentTypes.First(ct => ct.Id == d.dto.ContentVersionDto.ContentDto.ContentTypeId),
-                templates.FirstOrDefault(tem => tem.Id == (d.dto.TemplateId.HasValue ? d.dto.TemplateId.Value : -1)),
-                propertyData[d.dto.NodeId]));
-        }
-
-        /// <summary>
-        /// Private method to create a content object from a DocumentDto, which is used by Get and GetByVersion.
-        /// </summary>
-        /// <param name="dto"></param>
-        /// <param name="contentType"></param>
-        /// <param name="template"></param>
-        /// <param name="propCollection"></param>
-        /// <returns></returns>
-        private IContent CreateContentFromDto(DocumentDto dto,
-            IContentType contentType,
-            ITemplate template,
-            Models.PropertyCollection propCollection)
-        {
-            var factory = new ContentFactory(contentType, NodeObjectTypeId, dto.NodeId);
-            var content = factory.BuildEntity(dto);
-
-            //Check if template id is set on DocumentDto, and get ITemplate if it is.
-            if (dto.TemplateId.HasValue && dto.TemplateId.Value > 0)
+            for (var i = 0; i < dtos.Count; i++)
             {
-                content.Template = template ?? _templateRepository.Get(dto.TemplateId.Value);
-            }
-            else
-            {
-                //ensure there isn't one set.
-                content.Template = null;
+                var dto = dtos[i];
+
+                // if the cache contains the published version, use it
+                if (withCache)
+                {
+                    var cached = RuntimeCache.GetCacheItem<IContent>(GetCacheIdKey<IContent>(dto.NodeId));
+                    //only use this cached version if the dto returned is also the publish version, they must match
+                    if (cached != null && cached.Published && dto.Published)
+                    {
+                        content[i] = cached;
+                        continue;
+                    }
+                }
+
+                // else, need to fetch from the database
+                // content type repository is full-cache so OK to get each one independently
+                var contentType = _contentTypeRepository.Get(dto.ContentVersionDto.ContentDto.ContentTypeId);
+                var factory = new ContentFactory(contentType, NodeObjectTypeId, dto.NodeId);
+                content[i] = factory.BuildEntity(dto);
+
+                // need template
+                if (dto.TemplateId.HasValue && dto.TemplateId.Value > 0)
+                    templateIds.Add(dto.TemplateId.Value);
+
+                // need properties
+                defs.Add(new DocumentDefinition(
+                    dto.NodeId,
+                    dto.VersionId,
+                    dto.ContentVersionDto.VersionDate,
+                    dto.ContentVersionDto.ContentDto.NodeDto.CreateDate,
+                    contentType
+                ));
             }
 
-            content.Properties = propCollection;
+            // load all required templates in 1 query
+            var templates = _templateRepository.GetAll(templateIds.ToArray())
+                .ToDictionary(x => x.Id, x => x);
+
+            // load all properties for all documents from database in 1 query
+            var propertyData = GetPropertyCollection(sql, defs);
+
+            // assign
+            var dtoIndex = 0;
+            foreach (var def in defs)
+            {
+                // move to corresponding item (which has to exist)
+                while (dtos[dtoIndex].NodeId != def.Id) dtoIndex++;
+
+                // complete the item
+                var cc = content[dtoIndex];
+                var dto = dtos[dtoIndex];
+                ITemplate template = null;
+                if (dto.TemplateId.HasValue)
+                    templates.TryGetValue(dto.TemplateId.Value, out template); // else null
+                cc.Template = template;
+                cc.Properties = propertyData[cc.Id];
+
+                //on initial construction we don't want to have dirty properties tracked
+                // http://issues.umbraco.org/issue/U4-1946
+                ((Entity) cc).ResetDirtyProperties(false);
+            }
 
-            //on initial construction we don't want to have dirty properties tracked
-            // http://issues.umbraco.org/issue/U4-1946
-            ((Entity)content).ResetDirtyProperties(false);
             return content;
         }
 

src/Umbraco.Core/Persistence/Repositories/Interfaces/IMediaRepository.cs

@@ -38,5 +38,15 @@ namespace Umbraco.Core.Persistence.Repositories
         /// <returns>An Enumerable list of <see cref="IMedia"/> objects</returns>
         IEnumerable<IMedia> GetPagedResultsByQuery(IQuery<IMedia> query, long pageIndex, int pageSize, out long totalRecords,
             string orderBy, Direction orderDirection, bool orderBySystemField, string filter = "");
+
+        /// <summary>
+        /// Gets paged media descendants as XML by path
+        /// </summary>
+        /// <param name="path">Path starts with</param>
+        /// <param name="pageIndex">Page number</param>
+        /// <param name="pageSize">Page size</param>
+        /// <param name="totalRecords">Total records the query would return without paging</param>
+        /// <returns>A paged enumerable of XML entries of media items</returns>
+        IEnumerable<XElement> GetPagedXmlEntriesByPath(string path, long pageIndex, int pageSize, out long totalRecords);
     }
 }

src/Umbraco.Core/Persistence/Repositories/MediaRepository.cs

@@ -4,21 +4,17 @@ using System.Globalization;
 using System.Linq;
 using System.Text;
 using System.Xml.Linq;
-using Umbraco.Core.Configuration;
 using Umbraco.Core.Configuration.UmbracoSettings;
-using Umbraco.Core.Dynamics;
-using Umbraco.Core.IO;
 using Umbraco.Core.Logging;
 using Umbraco.Core.Models;
 using Umbraco.Core.Models.EntityBase;
 using Umbraco.Core.Models.Rdbms;
-
+using Umbraco.Core.Cache;
 using Umbraco.Core.Persistence.DatabaseModelDefinitions;
 using Umbraco.Core.Persistence.Factories;
 using Umbraco.Core.Persistence.Querying;
 using Umbraco.Core.Persistence.SqlSyntax;
 using Umbraco.Core.Persistence.UnitOfWork;
-using Umbraco.Core.Services;
 
 namespace Umbraco.Core.Persistence.Repositories
 {
@@ -137,6 +133,74 @@ namespace Umbraco.Core.Persistence.Repositories
 
         #region Overrides of VersionableRepositoryBase<IContent>
 
+        public override IEnumerable<IMedia> GetAllVersions(int id)
+        {
+            var sql = GetBaseQuery(false)
+                .Where(GetBaseWhereClause(), new { Id = id })
+                .OrderByDescending<ContentVersionDto>(x => x.VersionDate, SqlSyntax);
+            return ProcessQuery(sql, true);
+        }
+
+        private IEnumerable<IMedia> ProcessQuery(Sql sql, bool withCache = false)
+        {
+            // fetch returns a list so it's ok to iterate it in this method
+            var dtos = Database.Fetch<ContentVersionDto, ContentDto, NodeDto>(sql);
+            var content = new IMedia[dtos.Count];
+            var defs = new List<DocumentDefinition>();
+
+            for (var i = 0; i < dtos.Count; i++)
+            {
+                var dto = dtos[i];
+
+                // if the cache contains the item, use it
+                if (withCache)
+                {
+                    var cached = RuntimeCache.GetCacheItem<IMedia>(GetCacheIdKey<IMedia>(dto.NodeId));
+                    if (cached != null)
+                    {
+                        content[i] = cached;
+                        continue;
+                    }
+                }
+
+                // else, need to fetch from the database
+                // content type repository is full-cache so OK to get each one independently
+                var contentType = _mediaTypeRepository.Get(dto.ContentDto.ContentTypeId);
+                var factory = new MediaFactory(contentType, NodeObjectTypeId, dto.NodeId);
+                content[i] = factory.BuildEntity(dto);
+
+                // need properties
+                defs.Add(new DocumentDefinition(
+                    dto.NodeId,
+                    dto.VersionId,
+                    dto.VersionDate,
+                    dto.ContentDto.NodeDto.CreateDate,
+                    contentType
+                ));
+            }
+
+            // load all properties for all documents from database in 1 query
+            var propertyData = GetPropertyCollection(sql, defs);
+
+            // assign
+            var dtoIndex = 0;
+            foreach (var def in defs)
+            {
+                // move to corresponding item (which has to exist)
+                while (dtos[dtoIndex].NodeId != def.Id) dtoIndex++;
+
+                // complete the item
+                var cc = content[dtoIndex];
+                cc.Properties = propertyData[cc.Id];
+
+                //on initial construction we don't want to have dirty properties tracked
+                // http://issues.umbraco.org/issue/U4-1946
+                ((Entity) cc).ResetDirtyProperties(false);
+            }
+
+            return content;
+        }
+
         public override IMedia GetByVersion(Guid versionId)
         {
             var sql = GetBaseQuery(false);
@@ -163,90 +227,51 @@ namespace Umbraco.Core.Persistence.Repositories
             return media;
         }
 
-        public void RebuildXmlStructures(Func<IMedia, XElement> serializer, int groupSize = 5000, IEnumerable<int> contentTypeIds = null)
+        public void RebuildXmlStructures(Func<IMedia, XElement> serializer, int groupSize = 200, IEnumerable<int> contentTypeIds = null)
         {
+            // the previous way of doing this was to run it all in one big transaction,
+            // and to bulk-insert groups of xml rows - which works, until the transaction
+            // times out - and besides, because v7 transactions are ReadCommited, it does
+            // not bring much safety - so this reverts to updating each record individually,
+            // and it may be slower in the end, but should be more resilient.
 
-            //Ok, now we need to remove the data and re-insert it, we'll do this all in one transaction too.
-            using (var tr = Database.GetTransaction())
+            var baseId = 0;
+            var contentTypeIdsA = contentTypeIds == null ? new int[0] : contentTypeIds.ToArray();
+            while (true)
             {
-                //Remove all the data first, if anything fails after this it's no problem the transaction will be reverted
-                if (contentTypeIds == null)
-                {
-                    var mediaObjectType = Guid.Parse(Constants.ObjectTypes.Media);
-                    var subQuery = new Sql()
-                        .Select("DISTINCT cmsContentXml.nodeId")
-                        .From<ContentXmlDto>()
-                        .InnerJoin<NodeDto>()
-                        .On<ContentXmlDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                        .Where<NodeDto>(dto => dto.NodeObjectType == mediaObjectType);
+                // get the next group of nodes
+                var query = GetBaseQuery(false);
+                if (contentTypeIdsA.Length > 0)
+                    query = query
+                        .WhereIn<ContentDto>(x => x.ContentTypeId, contentTypeIdsA, SqlSyntax);
+                query = query
+                    .Where<NodeDto>(x => x.NodeId > baseId)
+                    .OrderBy<NodeDto>(x => x.NodeId, SqlSyntax);
+                var xmlItems = ProcessQuery(SqlSyntax.SelectTop(query, groupSize))
+                    .Select(x => new ContentXmlDto { NodeId = x.Id, Xml = serializer(x).ToString() })
+                    .ToList();
 
-                    var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                    Database.Execute(deleteSql);
-                }
-                else
+                // no more nodes, break
+                if (xmlItems.Count == 0) break;
+
+                foreach (var xmlItem in xmlItems)
                 {
-                    foreach (var id in contentTypeIds)
+                    try
                     {
-                        var id1 = id;
-                        var mediaObjectType = Guid.Parse(Constants.ObjectTypes.Media);
-                        var subQuery = new Sql()
-                            .Select("DISTINCT cmsContentXml.nodeId")
-                            .From<ContentXmlDto>()
-                            .InnerJoin<NodeDto>()
-                            .On<ContentXmlDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                            .InnerJoin<ContentDto>()
-                            .On<ContentDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                            .Where<NodeDto>(dto => dto.NodeObjectType == mediaObjectType)
-                            .Where<ContentDto>(dto => dto.ContentTypeId == id1);
-
-                        var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                        Database.Execute(deleteSql);
+                        // InsertOrUpdate tries to update first, which is good since it is what
+                        // should happen in most cases, then it tries to insert, and it should work
+                        // unless the node has been deleted, and we just report the exception
+                        Database.InsertOrUpdate(xmlItem);
+                    }
+                    catch (Exception e)
+                    {
+                        Logger.Error<MediaRepository>("Could not rebuild XML for nodeId=" + xmlItem.NodeId, e);
                     }
                 }
-
-                //now insert the data, again if something fails here, the whole transaction is reversed
-                if (contentTypeIds == null)
-                {
-                    var query = Query<IMedia>.Builder;
-                    RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                }
-                else
-                {
-                    foreach (var contentTypeId in contentTypeIds)
-                    {
-                        //copy local
-                        var id = contentTypeId;
-                        var query = Query<IMedia>.Builder.Where(x => x.ContentTypeId == id && x.Trashed == false);
-                        RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                    }
-                }
-
-                tr.Complete();
+                baseId = xmlItems.Last().NodeId;
             }
         }
 
-        private void RebuildXmlStructuresProcessQuery(Func<IMedia, XElement> serializer, IQuery<IMedia> query, Transaction tr, int pageSize)
-        {
-            var pageIndex = 0;
-            var total = long.MinValue;
-            var processed = 0;
-            do
-            {
-                var descendants = GetPagedResultsByQuery(query, pageIndex, pageSize, out total, "Path", Direction.Ascending, true);
-
-                var xmlItems = (from descendant in descendants
-                                let xml = serializer(descendant)
-                                select new ContentXmlDto { NodeId = descendant.Id, Xml = xml.ToDataString() }).ToArray();
-
-                //bulk insert it into the database
-                Database.BulkInsertRecords(xmlItems, tr);
-
-                processed += xmlItems.Length;
-
-                pageIndex++;
-            } while (processed < total);
-        }
-
         public void AddOrUpdateContentXml(IMedia content, Func<IMedia, XElement> xml)
         {
             _contentXmlRepository.AddOrUpdate(new ContentXmlEntity<IMedia>(content, xml));
@@ -299,6 +324,7 @@ namespace Umbraco.Core.Persistence.Repositories
 
             //Update with new correct path
             nodeDto.Path = string.Concat(parent.Path, ",", nodeDto.NodeId);
+            nodeDto.ValidatePathWithException();
             Database.Update(nodeDto);
 
             //Update entity with correct values
@@ -372,6 +398,7 @@ namespace Umbraco.Core.Persistence.Repositories
 
             //Updates the (base) node data - umbracoNode
             var nodeDto = dto.ContentDto.NodeDto;
+            nodeDto.ValidatePathWithException();
             var o = Database.Update(nodeDto);
 
             //Only update this DTO if the contentType has actually changed
@@ -470,65 +497,33 @@ namespace Umbraco.Core.Persistence.Repositories
 
             return GetPagedResultsByQuery<ContentVersionDto, Models.Media>(query, pageIndex, pageSize, out totalRecords,
                 new Tuple<string, string>("cmsContentVersion", "contentId"),
-                ProcessQuery, orderBy, orderDirection, orderBySystemField,
+                sql => ProcessQuery(sql), orderBy, orderDirection, orderBySystemField,
                 filterCallback);
 
         }
 
-        private IEnumerable<IMedia> ProcessQuery(Sql sql)
-        {
-            //NOTE: This doesn't allow properties to be part of the query
-            var dtos = Database.Fetch<ContentVersionDto, ContentDto, NodeDto>(sql);
-
-            var ids = dtos.Select(x => x.ContentDto.ContentTypeId).ToArray();
-
-            //content types
-            var contentTypes = ids.Length == 0 ? Enumerable.Empty<IMediaType>() : _mediaTypeRepository.GetAll(ids).ToArray();
-
-            var dtosWithContentTypes = dtos
-                //This select into and null check are required because we don't have a foreign damn key on the contentType column
-                // http://issues.umbraco.org/issue/U4-5503
-                .Select(x => new { dto = x, contentType = contentTypes.FirstOrDefault(ct => ct.Id == x.ContentDto.ContentTypeId) })
-                .Where(x => x.contentType != null)
-                .ToArray();
-
-            //Go get the property data for each document
-            var docDefs = dtosWithContentTypes.Select(d => new DocumentDefinition(
-                d.dto.NodeId,
-                d.dto.VersionId,
-                d.dto.VersionDate,
-                d.dto.ContentDto.NodeDto.CreateDate,
-                d.contentType))
-                .ToArray();
-
-            var propertyData = GetPropertyCollection(sql, docDefs);
-
-            return dtosWithContentTypes.Select(d => CreateMediaFromDto(
-                d.dto,
-                contentTypes.First(ct => ct.Id == d.dto.ContentDto.ContentTypeId),
-                propertyData[d.dto.NodeId]));
-        }
-
         /// <summary>
-        /// Private method to create a media object from a ContentDto
+        /// Gets paged media descendants as XML by path
         /// </summary>
-        /// <param name="d"></param>
-        /// <param name="contentType"></param>
-        /// <param name="propCollection"></param>
-        /// <returns></returns>
-        private IMedia CreateMediaFromDto(ContentVersionDto dto,
-            IMediaType contentType,
-            PropertyCollection propCollection)
+        /// <param name="path">Path starts with</param>
+        /// <param name="pageIndex">Page number</param>
+        /// <param name="pageSize">Page size</param>
+        /// <param name="totalRecords">Total records the query would return without paging</param>
+        /// <returns>A paged enumerable of XML entries of media items</returns>
+        public IEnumerable<XElement> GetPagedXmlEntriesByPath(string path, long pageIndex, int pageSize, out long totalRecords)
         {
-            var factory = new MediaFactory(contentType, NodeObjectTypeId, dto.NodeId);
-            var media = factory.BuildEntity(dto);
-
-            media.Properties = propCollection;
-
-            //on initial construction we don't want to have dirty properties tracked
-            // http://issues.umbraco.org/issue/U4-1946
-            ((Entity)media).ResetDirtyProperties(false);
-            return media;
+            Sql query;
+            if (path == "-1")
+            {
+                query = new Sql().Select("nodeId, xml").From("cmsContentXml").Where("nodeId IN (SELECT id FROM umbracoNode WHERE nodeObjectType = @0)", Guid.Parse(Constants.ObjectTypes.Media)).OrderBy("nodeId");
+            }
+            else
+            {
+                query = new Sql().Select("nodeId, xml").From("cmsContentXml").Where("nodeId IN (SELECT id FROM umbracoNode WHERE path LIKE @0)", path.EnsureEndsWith(",%")).OrderBy("nodeId");
+            }
+            var pagedResult = Database.Page<ContentXmlDto>(pageIndex+1, pageSize, query);
+            totalRecords = pagedResult.TotalItems;
+            return pagedResult.Items.Select(dto => XElement.Parse(dto.Xml));
         }
 
         /// <summary>

src/Umbraco.Core/Persistence/Repositories/MemberRepository.cs

@@ -2,24 +2,19 @@
 using System.Collections.Generic;
 using System.Globalization;
 using System.Linq;
-using System.Linq.Expressions;
 using System.Text;
 using System.Xml.Linq;
-using Umbraco.Core.Configuration;
 using Umbraco.Core.Configuration.UmbracoSettings;
-using Umbraco.Core.IO;
 using Umbraco.Core.Logging;
 using Umbraco.Core.Models.EntityBase;
 using Umbraco.Core.Models;
 using Umbraco.Core.Models.Rdbms;
-
+using Umbraco.Core.Cache;
 using Umbraco.Core.Persistence.DatabaseModelDefinitions;
 using Umbraco.Core.Persistence.Factories;
 using Umbraco.Core.Persistence.Querying;
-using Umbraco.Core.Persistence.Relators;
 using Umbraco.Core.Persistence.SqlSyntax;
 using Umbraco.Core.Persistence.UnitOfWork;
-using Umbraco.Core.Dynamics;
 
 namespace Umbraco.Core.Persistence.Repositories
 {
@@ -380,88 +375,57 @@ namespace Umbraco.Core.Persistence.Repositories
 
         #region Overrides of VersionableRepositoryBase<IMembershipUser>
 
-        public void RebuildXmlStructures(Func<IMember, XElement> serializer, int groupSize = 5000, IEnumerable<int> contentTypeIds = null)
+        public override IEnumerable<IMember> GetAllVersions(int id)
         {
-
-            //Ok, now we need to remove the data and re-insert it, we'll do this all in one transaction too.
-            using (var tr = Database.GetTransaction())
-            {
-                //Remove all the data first, if anything fails after this it's no problem the transaction will be reverted
-                if (contentTypeIds == null)
-                {
-                    var memberObjectType = Guid.Parse(Constants.ObjectTypes.Member);
-                    var subQuery = new Sql()
-                        .Select("DISTINCT cmsContentXml.nodeId")
-                        .From<ContentXmlDto>()
-                        .InnerJoin<NodeDto>()
-                        .On<ContentXmlDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                        .Where<NodeDto>(dto => dto.NodeObjectType == memberObjectType);
-
-                    var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                    Database.Execute(deleteSql);
-                }
-                else
-                {
-                    foreach (var id in contentTypeIds)
-                    {
-                        var id1 = id;
-                        var memberObjectType = Guid.Parse(Constants.ObjectTypes.Member);
-                        var subQuery = new Sql()
-                            .Select("DISTINCT cmsContentXml.nodeId")
-                            .From<ContentXmlDto>()
-                            .InnerJoin<NodeDto>()
-                            .On<ContentXmlDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                            .InnerJoin<ContentDto>()
-                            .On<ContentDto, NodeDto>(left => left.NodeId, right => right.NodeId)
-                            .Where<NodeDto>(dto => dto.NodeObjectType == memberObjectType)
-                            .Where<ContentDto>(dto => dto.ContentTypeId == id1);
-
-                        var deleteSql = SqlSyntax.GetDeleteSubquery("cmsContentXml", "nodeId", subQuery);
-                        Database.Execute(deleteSql);
-                    }
-                }
-
-                //now insert the data, again if something fails here, the whole transaction is reversed
-                if (contentTypeIds == null)
-                {
-                    var query = Query<IMember>.Builder;
-                    RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                }
-                else
-                {
-                    foreach (var contentTypeId in contentTypeIds)
-                    {
-                        //copy local
-                        var id = contentTypeId;
-                        var query = Query<IMember>.Builder.Where(x => x.ContentTypeId == id && x.Trashed == false);
-                        RebuildXmlStructuresProcessQuery(serializer, query, tr, groupSize);
-                    }
-                }
-
-                tr.Complete();
-            }
+            var sql = GetBaseQuery(false)
+                .Where(GetBaseWhereClause(), new { Id = id })
+                .OrderByDescending<ContentVersionDto>(x => x.VersionDate, SqlSyntax);
+            return ProcessQuery(sql, true);
         }
 
-        private void RebuildXmlStructuresProcessQuery(Func<IMember, XElement> serializer, IQuery<IMember> query, Transaction tr, int pageSize)
+        public void RebuildXmlStructures(Func<IMember, XElement> serializer, int groupSize = 200, IEnumerable<int> contentTypeIds = null)
         {
-            var pageIndex = 0;
-            var total = long.MinValue;
-            var processed = 0;
-            do
+            // the previous way of doing this was to run it all in one big transaction,
+            // and to bulk-insert groups of xml rows - which works, until the transaction
+            // times out - and besides, because v7 transactions are ReadCommited, it does
+            // not bring much safety - so this reverts to updating each record individually,
+            // and it may be slower in the end, but should be more resilient.
+
+            var baseId = 0;
+            var contentTypeIdsA = contentTypeIds == null ? new int[0] : contentTypeIds.ToArray();
+            while (true)
             {
-                var descendants = GetPagedResultsByQuery(query, pageIndex, pageSize, out total, "Path", Direction.Ascending, true);
+                // get the next group of nodes
+                var query = GetBaseQuery(false);
+                if (contentTypeIdsA.Length > 0)
+                    query = query
+                        .WhereIn<ContentDto>(x => x.ContentTypeId, contentTypeIdsA, SqlSyntax);
+                query = query
+                    .Where<NodeDto>(x => x.NodeId > baseId)
+                    .OrderBy<NodeDto>(x => x.NodeId, SqlSyntax);
+                var xmlItems = ProcessQuery(SqlSyntax.SelectTop(query, groupSize))
+                    .Select(x => new ContentXmlDto { NodeId = x.Id, Xml = serializer(x).ToString() })
+                    .ToList();
 
-                var xmlItems = (from descendant in descendants
-                                let xml = serializer(descendant)
-                                select new ContentXmlDto { NodeId = descendant.Id, Xml = xml.ToDataString() }).ToArray();
+                // no more nodes, break
+                if (xmlItems.Count == 0) break;
 
-                //bulk insert it into the database
-                Database.BulkInsertRecords(xmlItems, tr);
-
-                processed += xmlItems.Length;
-
-                pageIndex++;
-            } while (processed < total);
+                foreach (var xmlItem in xmlItems)
+                {
+                    try
+                    {
+                        // InsertOrUpdate tries to update first, which is good since it is what
+                        // should happen in most cases, then it tries to insert, and it should work
+                        // unless the node has been deleted, and we just report the exception
+                        Database.InsertOrUpdate(xmlItem);
+                    }
+                    catch (Exception e)
+                    {
+                        Logger.Error<MediaRepository>("Could not rebuild XML for nodeId=" + xmlItem.NodeId, e);
+                    }
+                }
+                baseId = xmlItems.Last().NodeId;
+            }
         }
 
         public override IMember GetByVersion(Guid versionId)
@@ -655,7 +619,7 @@ namespace Umbraco.Core.Persistence.Repositories
 
             return GetPagedResultsByQuery<MemberDto, Member>(query, pageIndex, pageSize, out totalRecords,
                 new Tuple<string, string>("cmsMember", "nodeId"),
-                ProcessQuery, orderBy, orderDirection, orderBySystemField,
+                sql => ProcessQuery(sql), orderBy, orderDirection, orderBySystemField,
                 filterCallback);
         }
 
@@ -695,59 +659,65 @@ namespace Umbraco.Core.Persistence.Repositories
             return base.GetEntityPropertyNameForOrderBy(orderBy);
         }
 
-        private IEnumerable<IMember> ProcessQuery(Sql sql)
+        private IEnumerable<IMember> ProcessQuery(Sql sql, bool withCache = false)
         {
-            //NOTE: This doesn't allow properties to be part of the query
+            // fetch returns a list so it's ok to iterate it in this method
             var dtos = Database.Fetch<MemberDto, ContentVersionDto, ContentDto, NodeDto>(sql);
 
-            var ids = dtos.Select(x => x.ContentVersionDto.ContentDto.ContentTypeId).ToArray();
+            var content = new IMember[dtos.Count];
+            var defs = new List<DocumentDefinition>();
 
-            //content types
-            var contentTypes = ids.Length == 0 ? Enumerable.Empty<IMemberType>() : _memberTypeRepository.GetAll(ids).ToArray();
+            for (var i = 0; i < dtos.Count; i++)
+            {
+                var dto = dtos[i];
 
-            var dtosWithContentTypes = dtos
-                //This select into and null check are required because we don't have a foreign damn key on the contentType column
-                // http://issues.umbraco.org/issue/U4-5503
-                .Select(x => new { dto = x, contentType = contentTypes.FirstOrDefault(ct => ct.Id == x.ContentVersionDto.ContentDto.ContentTypeId) })
-                .Where(x => x.contentType != null)
-                .ToArray();
+                // if the cache contains the item, use it
+                if (withCache)
+                {
+                    var cached = RuntimeCache.GetCacheItem<IMember>(GetCacheIdKey<IMember>(dto.NodeId));
+                    if (cached != null)
+                    {
+                        content[i] = cached;
+                        continue;
+                    }
+                }
 
-            //Go get the property data for each document
-            IEnumerable<DocumentDefinition> docDefs = dtosWithContentTypes.Select(d => new DocumentDefinition(
-                d.dto.NodeId,
-                d.dto.ContentVersionDto.VersionId,
-                d.dto.ContentVersionDto.VersionDate,
-                d.dto.ContentVersionDto.ContentDto.NodeDto.CreateDate,
-                d.contentType));
+                // else, need to fetch from the database
+                // content type repository is full-cache so OK to get each one independently
+                var contentType = _memberTypeRepository.Get(dto.ContentVersionDto.ContentDto.ContentTypeId);
+                var factory = new MemberFactory(contentType, NodeObjectTypeId, dto.NodeId);
+                content[i] = factory.BuildEntity(dto);
 
-            var propertyData = GetPropertyCollection(sql, docDefs);
+                // need properties
+                defs.Add(new DocumentDefinition(
+                    dto.NodeId,
+                    dto.ContentVersionDto.VersionId,
+                    dto.ContentVersionDto.VersionDate,
+                    dto.ContentVersionDto.ContentDto.NodeDto.CreateDate,
+                    contentType
+                ));
+            }
 
-            return dtosWithContentTypes.Select(d => CreateMemberFromDto(
-                        d.dto,
-                        contentTypes.First(ct => ct.Id == d.dto.ContentVersionDto.ContentDto.ContentTypeId),
-                        propertyData[d.dto.NodeId]));
-        }
+            // load all properties for all documents from database in 1 query
+            var propertyData = GetPropertyCollection(sql, defs);
 
-        /// <summary>
-        /// Private method to create a member object from a MemberDto
-        /// </summary>
-        /// <param name="dto"></param>
-        /// <param name="contentType"></param>
-        /// <param name="propCollection"></param>
-        /// <returns></returns>
-        private IMember CreateMemberFromDto(MemberDto dto,
-            IMemberType contentType,
-            PropertyCollection propCollection)
-        {
-            var factory = new MemberFactory(contentType, NodeObjectTypeId, dto.ContentVersionDto.NodeId);
-            var member = factory.BuildEntity(dto);
+            // assign
+            var dtoIndex = 0;
+            foreach (var def in defs)
+            {
+                // move to corresponding item (which has to exist)
+                while (dtos[dtoIndex].NodeId != def.Id) dtoIndex++;
 
-            member.Properties = propCollection;
+                // complete the item
+                var cc = content[dtoIndex];
+                cc.Properties = propertyData[cc.Id];
 
-            //on initial construction we don't want to have dirty properties tracked
-            // http://issues.umbraco.org/issue/U4-1946
-            ((Entity)member).ResetDirtyProperties(false);
-            return member;
+                //on initial construction we don't want to have dirty properties tracked
+                // http://issues.umbraco.org/issue/U4-1946
+                ((Entity)cc).ResetDirtyProperties(false);
+            }
+
+            return content;
         }
 
         /// <summary>

src/Umbraco.Core/Persistence/Repositories/RelationRepository.cs

@@ -42,34 +42,17 @@ namespace Umbraco.Core.Persistence.Repositories
                 throw new Exception(string.Format("RelationType with Id: {0} doesn't exist", dto.RelationType));
 
             var factory = new RelationFactory(relationType);
-            var entity = factory.BuildEntity(dto);
-
-            //on initial construction we don't want to have dirty properties tracked
-            // http://issues.umbraco.org/issue/U4-1946
-            ((TracksChangesEntityBase)entity).ResetDirtyProperties(false);
-
-            return entity;
+            return DtoToEntity(dto, factory);
         }
 
-        //TODO: Fix N+1 !
-
         protected override IEnumerable<IRelation> PerformGetAll(params int[] ids)
         {
-            if (ids.Any())
-            {
-                foreach (var id in ids)
-                {
-                    yield return Get(id);
-                }
-            }
-            else
-            {
-                var dtos = Database.Fetch<RelationDto>("WHERE id > 0");
-                foreach (var dto in dtos)
-                {
-                    yield return Get(dto.Id);
-                }
-            }
+            var sql = GetBaseQuery(false);
+            if (ids.Length > 0)
+                sql.WhereIn<RelationDto>(x => x.Id, ids);
+            sql.OrderBy<RelationDto>(x => x.RelationType);
+            var dtos = Database.Fetch<RelationDto>(sql);
+            return DtosToEntities(dtos);
         }
 
         protected override IEnumerable<IRelation> PerformGetByQuery(IQuery<IRelation> query)
@@ -77,13 +60,36 @@ namespace Umbraco.Core.Persistence.Repositories
             var sqlClause = GetBaseQuery(false);
             var translator = new SqlTranslator<IRelation>(sqlClause, query);
             var sql = translator.Translate();
-
+            sql.OrderBy<RelationDto>(x => x.RelationType);
             var dtos = Database.Fetch<RelationDto>(sql);
+            return DtosToEntities(dtos);
+        }
 
-            foreach (var dto in dtos)
+        private IEnumerable<IRelation> DtosToEntities(IEnumerable<RelationDto> dtos)
+        {
+            // in most cases, the relation type will be the same for all of them,
+            // plus we've ordered the relations by type, so try to allocate as few
+            // factories as possible - bearing in mind that relation types are cached
+            RelationFactory factory = null;
+            var relationTypeId = -1;
+
+            return dtos.Select(x =>
             {
-                yield return Get(dto.Id);
-            }
+                if (relationTypeId != x.RelationType)
+                    factory = new RelationFactory(_relationTypeRepository.Get(relationTypeId = x.RelationType));
+                return DtoToEntity(x, factory);
+            });
+        }
+
+        private static IRelation DtoToEntity(RelationDto dto, RelationFactory factory)
+        {
+            var entity = factory.BuildEntity(dto);
+
+            //on initial construction we don't want to have dirty properties tracked
+            // http://issues.umbraco.org/issue/U4-1946
+            ((TracksChangesEntityBase)entity).ResetDirtyProperties(false);
+
+            return entity;
         }
 
         #endregion

src/Umbraco.Core/Persistence/Repositories/RelationTypeRepository.cs

@@ -1,6 +1,7 @@
 using System;
 using System.Collections.Generic;
 using System.Linq;
+using Umbraco.Core.Cache;
 using Umbraco.Core.Logging;
 using Umbraco.Core.Models;
 using Umbraco.Core.Models.EntityBase;
@@ -18,50 +19,42 @@ namespace Umbraco.Core.Persistence.Repositories
     /// </summary>
     internal class RelationTypeRepository : PetaPocoRepositoryBase<int, IRelationType>, IRelationTypeRepository
     {
-
         public RelationTypeRepository(IDatabaseUnitOfWork work, CacheHelper cache, ILogger logger, ISqlSyntaxProvider sqlSyntax)
             : base(work, cache, logger, sqlSyntax)
+        { }
+
+        // assuming we don't have tons of relation types, use a FullDataSet policy, ie
+        // cache the entire GetAll result once in a single collection - which can expire
+        private FullDataSetRepositoryCachePolicyFactory<IRelationType, int> _cachePolicyFactory;
+        protected override IRepositoryCachePolicyFactory<IRelationType, int> CachePolicyFactory
         {
+            get
+            {
+                return _cachePolicyFactory 
+                    ?? (_cachePolicyFactory = new FullDataSetRepositoryCachePolicyFactory<IRelationType, int>(
+                        RuntimeCache, GetEntityId, () => PerformGetAll(), expires: true));
+            }
         }
 
         #region Overrides of RepositoryBase<int,RelationType>
 
         protected override IRelationType PerformGet(int id)
         {
-            var sql = GetBaseQuery(false);
-            sql.Where(GetBaseWhereClause(), new { Id = id });
-
-            var dto = Database.Fetch<RelationTypeDto>(SqlSyntax.SelectTop(sql, 1)).FirstOrDefault();
-            if (dto == null)
-                return null;
-
-            var factory = new RelationTypeFactory();
-            var entity = factory.BuildEntity(dto);
-
-            //on initial construction we don't want to have dirty properties tracked
-            // http://issues.umbraco.org/issue/U4-1946
-            ((TracksChangesEntityBase)entity).ResetDirtyProperties(false);
-
-            return entity;
+            // use the underlying GetAll which will force cache all content types
+            return GetAll().FirstOrDefault(x => x.Id == id);
         }
 
         protected override IEnumerable<IRelationType> PerformGetAll(params int[] ids)
         {
+            var sql = GetBaseQuery(false);
+
+            // should not happen due to the cache policy
             if (ids.Any())
-            {
-                foreach (var id in ids)
-                {
-                    yield return Get(id);
-                }
-            }
-            else
-            {
-                var dtos = Database.Fetch<RelationTypeDto>("WHERE id > 0");
-                foreach (var dto in dtos)
-                {
-                    yield return Get(dto.Id);
-                }
-            }
+                throw new NotImplementedException();
+
+            var dtos = Database.Fetch<RelationTypeDto>(sql);
+            var factory = new RelationTypeFactory();
+            return dtos.Select(x => DtoToEntity(x, factory));
         }
 
         protected override IEnumerable<IRelationType> PerformGetByQuery(IQuery<IRelationType> query)
@@ -71,11 +64,19 @@ namespace Umbraco.Core.Persistence.Repositories
             var sql = translator.Translate();
 
             var dtos = Database.Fetch<RelationTypeDto>(sql);
+            var factory = new RelationTypeFactory();
+            return dtos.Select(x => DtoToEntity(x, factory));
+        }
 
-            foreach (var dto in dtos)
-            {
-                yield return Get(dto.Id);
-            }
+        private static IRelationType DtoToEntity(RelationTypeDto dto, RelationTypeFactory factory)
+        {
+            var entity = factory.BuildEntity(dto);
+
+            //on initial construction we don't want to have dirty properties tracked
+            // http://issues.umbraco.org/issue/U4-1946
+            ((TracksChangesEntityBase) entity).ResetDirtyProperties(false);
+
+            return entity;
         }
 
         #endregion

src/Umbraco.Core/Persistence/Repositories/RepositoryBase.cs

@@ -83,6 +83,12 @@ namespace Umbraco.Core.Persistence.Repositories
         }
 
 
+        #region Static Queries
+
+        private IQuery<TEntity> _hasIdQuery;
+
+        #endregion
+
         protected virtual TId GetEntityId(TEntity entity)
         {
             return (TId)(object)entity.Id;
@@ -111,9 +117,14 @@ namespace Umbraco.Core.Persistence.Repositories
                     RuntimeCache,
                     new RepositoryCachePolicyOptions(() =>
                     {
+                        //create it once if it is needed (no need for locking here)
+                        if (_hasIdQuery == null)
+                        {
+                            _hasIdQuery = Query<TEntity>.Builder.Where(x => x.Id != 0);
+                        }
+
                         //Get count of all entities of current type (TEntity) to ensure cached result is correct
-                        var query = Query<TEntity>.Builder.Where(x => x.Id != 0);
-                        return PerformCount(query);
+                        return PerformCount(_hasIdQuery);
                     })));
             }
         }

src/Umbraco.Core/Persistence/Repositories/TemplateRepository.cs

@@ -241,7 +241,12 @@ namespace Umbraco.Core.Persistence.Repositories
                 {
                     entity.Path = string.Concat(parent.Path, ",", entity.Id);
                 }
-
+                else
+                {
+                    //this means that the master template has been removed, so we need to reset the template's
+                    //path to be at the root
+                    entity.Path = string.Concat("-1,", entity.Id);
+                }
             }
 
             //Get TemplateDto from db to get the Primary key of the entity

src/Umbraco.Core/Persistence/RepositoryFactory.cs

@@ -197,7 +197,7 @@ namespace Umbraco.Core.Persistence
         {
             return new RelationRepository(
                 uow,
-                _noCache, //never cache
+                _noCache,
                 _logger, _sqlSyntax,
                 CreateRelationTypeRepository(uow));
         }
@@ -206,7 +206,7 @@ namespace Umbraco.Core.Persistence
         {
             return new RelationTypeRepository(
                 uow,
-                _noCache, //never cache
+                _cacheHelper,
                 _logger, _sqlSyntax);
         }
 
@@ -343,4 +343,4 @@ namespace Umbraco.Core.Persistence
                 _sqlSyntax);
         }
     }
-}
+}

src/Umbraco.Core/Persistence/SqlSyntax/DbTypes.cs

@@ -4,6 +4,7 @@ using System.Data;
 
 namespace Umbraco.Core.Persistence.SqlSyntax
 {
+    //TODO: TSyntax should be removed, it's not used/needed here
     public class DbTypes<TSyntax>
         where TSyntax : ISqlSyntaxProvider
     {

src/Umbraco.Core/Persistence/SqlSyntax/MicrosoftSqlSyntaxProviderBase.cs

@@ -1,4 +1,6 @@
 using System;
+using System.Data;
+using System.Linq;
 using Umbraco.Core.Persistence.Querying;
 
 namespace Umbraco.Core.Persistence.SqlSyntax
@@ -133,5 +135,111 @@ namespace Umbraco.Core.Persistence.SqlSyntax
                     throw new ArgumentOutOfRangeException("columnType");
             }
         }
+
+        /// <summary>
+        /// This uses a the DbTypeMap created and custom mapping to resolve the SqlDbType
+        /// </summary>
+        /// <param name="clrType"></param>
+        /// <returns></returns>
+        public virtual SqlDbType GetSqlDbType(Type clrType)
+        {
+            var dbType = DbTypeMap.ColumnDbTypeMap.First(x => x.Key == clrType).Value;
+            return GetSqlDbType(dbType);
+        }
+
+        /// <summary>
+        /// Returns the mapped SqlDbType for the DbType specified
+        /// </summary>
+        /// <param name="dbType"></param>
+        /// <returns></returns>
+        public virtual SqlDbType GetSqlDbType(DbType dbType)
+        {
+            var sqlDbType = SqlDbType.NVarChar;
+
+            //SEE: https://msdn.microsoft.com/en-us/library/cc716729(v=vs.110).aspx
+            // and https://msdn.microsoft.com/en-us/library/yy6y35y8%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396
+            switch (dbType)
+            {
+                case DbType.AnsiString:
+                    sqlDbType = SqlDbType.VarChar;
+                    break;
+                case DbType.Binary:
+                    sqlDbType = SqlDbType.VarBinary;
+                    break;
+                case DbType.Byte:
+                    sqlDbType = SqlDbType.TinyInt;
+                    break;
+                case DbType.Boolean:
+                    sqlDbType = SqlDbType.Bit;
+                    break;
+                case DbType.Currency:
+                    sqlDbType = SqlDbType.Money;
+                    break;
+                case DbType.Date:
+                    sqlDbType = SqlDbType.Date;
+                    break;
+                case DbType.DateTime:
+                    sqlDbType = SqlDbType.DateTime;
+                    break;
+                case DbType.Decimal:
+                    sqlDbType = SqlDbType.Decimal;
+                    break;
+                case DbType.Double:
+                    sqlDbType = SqlDbType.Float;
+                    break;
+                case DbType.Guid:
+                    sqlDbType = SqlDbType.UniqueIdentifier;
+                    break;
+                case DbType.Int16:
+                    sqlDbType = SqlDbType.SmallInt;
+                    break;
+                case DbType.Int32:
+                    sqlDbType = SqlDbType.Int;
+                    break;
+                case DbType.Int64:
+                    sqlDbType = SqlDbType.BigInt;
+                    break;
+                case DbType.Object:
+                    sqlDbType = SqlDbType.Variant;
+                    break;
+                case DbType.SByte:
+                    throw new NotSupportedException("Inferring a SqlDbType from SByte is not supported.");
+                case DbType.Single:
+                    sqlDbType = SqlDbType.Real;
+                    break;
+                case DbType.String:
+                    sqlDbType = SqlDbType.NVarChar;
+                    break;
+                case DbType.Time:
+                    sqlDbType = SqlDbType.Time;
+                    break;
+                case DbType.UInt16:
+                    throw new NotSupportedException("Inferring a SqlDbType from UInt16 is not supported.");
+                case DbType.UInt32:
+                    throw new NotSupportedException("Inferring a SqlDbType from UInt32 is not supported.");
+                case DbType.UInt64:
+                    throw new NotSupportedException("Inferring a SqlDbType from UInt64 is not supported.");
+                case DbType.VarNumeric:
+                    throw new NotSupportedException("Inferring a VarNumeric from UInt64 is not supported.");
+                case DbType.AnsiStringFixedLength:
+                    sqlDbType = SqlDbType.Char;
+                    break;
+                case DbType.StringFixedLength:
+                    sqlDbType = SqlDbType.NChar;
+                    break;
+                case DbType.Xml:
+                    sqlDbType = SqlDbType.Xml;
+                    break;
+                case DbType.DateTime2:
+                    sqlDbType = SqlDbType.DateTime2;
+                    break;
+                case DbType.DateTimeOffset:
+                    sqlDbType = SqlDbType.DateTimeOffset;
+                    break;
+                default:
+                    throw new ArgumentOutOfRangeException();
+            }
+            return sqlDbType;
+        }
     }
 }

src/Umbraco.Core/Persistence/SqlSyntax/SqlServerSyntaxProvider.cs

@@ -6,16 +6,11 @@ using Umbraco.Core.Persistence.DatabaseModelDefinitions;
 namespace Umbraco.Core.Persistence.SqlSyntax
 {
     /// <summary>
-    /// Represents an SqlSyntaxProvider for Sql Server
+    /// Represents an SqlSyntaxProvider for Sql Server.
     /// </summary>
-    [SqlSyntaxProviderAttribute(Constants.DatabaseProviders.SqlServer)]
+    [SqlSyntaxProvider(Constants.DatabaseProviders.SqlServer)]
     public class SqlServerSyntaxProvider : MicrosoftSqlSyntaxProviderBase<SqlServerSyntaxProvider>
     {
-        public SqlServerSyntaxProvider()
-        {
-            
-        }     
-
         /// <summary>
         /// Gets/sets the version of the current SQL server instance
         /// </summary>
@@ -31,7 +26,7 @@ namespace Umbraco.Core.Persistence.SqlSyntax
                 switch (firstPart)
                 {
                     case "13":
-                        _versionName = SqlServerVersionName.V2014;
+                        _versionName = SqlServerVersionName.V2016;
                         break;
                     case "12":
                         _versionName = SqlServerVersionName.V2014;
@@ -75,7 +70,7 @@ namespace Umbraco.Core.Persistence.SqlSyntax
         {
             var items = db.Fetch<dynamic>("SELECT TableName = t.Name,ColumnName = c.Name,dc.Name,dc.[Definition] FROM sys.tables t INNER JOIN sys.default_constraints dc ON t.object_id = dc.parent_object_id INNER JOIN sys.columns c ON dc.parent_object_id = c.object_id AND c.column_id = dc.parent_column_id");
             return items.Select(x => new Tuple<string, string, string, string>(x.TableName, x.ColumnName, x.Name, x.Definition));
-        } 
+        }
 
         public override IEnumerable<string> GetTablesInSchema(Database db)
         {
@@ -120,9 +115,9 @@ from sys.tables as T inner join sys.indexes as I on T.[object_id] = I.[object_id
    inner join sys.all_columns as AC on IC.[object_id] = AC.[object_id] and IC.[column_id] = AC.[column_id] 
 WHERE I.name NOT LIKE 'PK_%'
 order by T.name, I.name");
-            return items.Select(item => new Tuple<string, string, string, bool>(item.TABLE_NAME, item.INDEX_NAME, item.COLUMN_NAME, 
+            return items.Select(item => new Tuple<string, string, string, bool>(item.TABLE_NAME, item.INDEX_NAME, item.COLUMN_NAME,
                 item.UNIQUE == 1)).ToList();
-            
+
         }
 
         public override bool DoesTableExist(Database db, string tableName)
@@ -164,7 +159,7 @@ order by T.name, I.name");
             switch (systemMethod)
             {
                 case SystemMethods.NewGuid:
-                    return "NEWID()";                
+                    return "NEWID()";
                 case SystemMethods.CurrentDateTime:
                     return "GETDATE()";
                 //case SystemMethods.NewSequentialId:
@@ -181,11 +176,11 @@ order by T.name, I.name");
             get { return "ALTER TABLE [{0}] DROP CONSTRAINT [DF_{0}_{1}]"; }
         }
 
-        
+
         public override string DropIndex { get { return "DROP INDEX {0} ON {1}"; } }
 
         public override string RenameColumn { get { return "sp_rename '{0}.{1}', '{2}', 'COLUMN'"; } }
 
-        
+
     }
 }

src/Umbraco.Core/Properties/AssemblyInfo.cs

@@ -32,6 +32,7 @@ using System.Security.Permissions;
 
 [assembly: InternalsVisibleTo("umbraco.editorControls")]
 [assembly: InternalsVisibleTo("Umbraco.Tests")]
+[assembly: InternalsVisibleTo("Umbraco.Tests.Benchmarks")]
 [assembly: InternalsVisibleTo("Umbraco.Core")]
 [assembly: InternalsVisibleTo("Umbraco.Web")]
 [assembly: InternalsVisibleTo("Umbraco.Web.UI")]

src/Umbraco.Core/Security/BackOfficeUserStore.cs

@@ -644,6 +644,13 @@ namespace Umbraco.Core.Security
             {
                 anythingChanged = true;
                 user.IsLockedOut = identityUser.IsLockedOut;
+
+                if (user.IsLockedOut)
+                {
+                    //need to set the last lockout date
+                    user.LastLockoutDate = DateTime.Now;
+                }
+
             }
             if (user.Username != identityUser.UserName && identityUser.UserName.IsNullOrWhiteSpace() == false)
             {

src/Umbraco.Core/Services/ContentService.cs

@@ -2,6 +2,7 @@ using System;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Globalization;
+using System.IO;
 using System.Linq;
 using System.Threading;
 using System.Xml;
@@ -11,6 +12,7 @@ using Umbraco.Core.Configuration;
 using Umbraco.Core.Events;
 using Umbraco.Core.Logging;
 using Umbraco.Core.Models;
+using Umbraco.Core.Models.EntityBase;
 using Umbraco.Core.Models.Membership;
 using Umbraco.Core.Models.Rdbms;
 using Umbraco.Core.Persistence;
@@ -56,6 +58,12 @@ namespace Umbraco.Core.Services
             _userService = userService;
         }
 
+        #region Static Queries
+
+        private IQuery<IContent> _notTrashedQuery;
+
+        #endregion
+
         public int CountPublished(string contentTypeAlias = null)
         {
             var uow = UowProvider.GetUnitOfWork();
@@ -708,7 +716,12 @@ namespace Umbraco.Core.Services
         /// <returns>An Enumerable list of <see cref="IContent"/> objects</returns>
         public IEnumerable<IContent> GetDescendants(IContent content)
         {
-            using (var repository = RepositoryFactory.CreateContentRepository(UowProvider.GetUnitOfWork()))
+            //This is a check to ensure that the path is correct for this entity to avoid problems like: http://issues.umbraco.org/issue/U4-9336 due to data corruption
+            if (content.ValidatePath() == false)
+                throw new InvalidDataException(string.Format("The content item {0} has an invalid path: {1} with parentID: {2}", content.Id, content.Path, content.ParentId));
+
+            var uow = UowProvider.GetUnitOfWork();
+            using (var repository = RepositoryFactory.CreateContentRepository(uow))
             {
                 var pathMatch = content.Path + ",";
                 var query = Query<IContent>.Builder.Where(x => x.Path.StartsWith(pathMatch) && x.Id != content.Id);
@@ -787,10 +800,15 @@ namespace Umbraco.Core.Services
         /// <returns></returns>
         internal IEnumerable<IContent> GetAllPublished()
         {
+            //create it once if it is needed (no need for locking here)
+            if (_notTrashedQuery == null)
+            {
+                _notTrashedQuery = Query<IContent>.Builder.Where(x => x.Trashed == false);
+            }
+
             using (var repository = RepositoryFactory.CreateContentRepository(UowProvider.GetUnitOfWork()))
             {
-                var query = Query<IContent>.Builder.Where(x => x.Trashed == false);
-                return repository.GetByPublishedVersion(query);
+                return repository.GetByPublishedVersion(_notTrashedQuery);
             }
         }
 
@@ -985,6 +1003,10 @@ namespace Umbraco.Core.Services
 
             using (new WriteLock(Locker))
             {
+                //Hack: this ensures that the entity's path is valid and if not it fixes/persists it
+                //see: http://issues.umbraco.org/issue/U4-9336
+                content.EnsureValidPath(Logger, entity => GetById(entity.ParentId), QuickUpdate);
+
                 var originalPath = content.Path;
 
                 if (Trashing.IsRaisedEventCancelled(
@@ -1672,16 +1694,16 @@ namespace Umbraco.Core.Services
         /// <returns>True if sorting succeeded, otherwise False</returns>
         public bool Sort(IEnumerable<IContent> items, int userId = 0, bool raiseEvents = true)
         {
+            var asArray = items.ToArray();
             if (raiseEvents)
             {
-                if (Saving.IsRaisedEventCancelled(new SaveEventArgs<IContent>(items), this))
+                if (Saving.IsRaisedEventCancelled(new SaveEventArgs<IContent>(asArray), this))
                     return false;
             }
 
             var shouldBePublished = new List<IContent>();
             var shouldBeSaved = new List<IContent>();
-
-            var asArray = items.ToArray();
+            
             using (new WriteLock(Locker))
             {
                 var uow = UowProvider.GetUnitOfWork();
@@ -1801,6 +1823,23 @@ namespace Umbraco.Core.Services
 
         #region Private Methods
 
+        /// <summary>
+        /// Hack: This is used to fix some data if an entity's properties are invalid/corrupt
+        /// </summary>
+        /// <param name="content"></param>
+        private void QuickUpdate(IContent content)
+        {
+            if (content == null) throw new ArgumentNullException("content");
+            if (content.HasIdentity == false) throw new InvalidOperationException("Cannot update an entity without an Identity");
+
+            var uow = UowProvider.GetUnitOfWork();
+            using (var repository = RepositoryFactory.CreateContentRepository(uow))
+            {
+                repository.AddOrUpdate(content);                
+                uow.Commit();
+            }
+        }
+
         private void Audit(AuditType type, string message, int userId, int objectId)
         {
             var uow = UowProvider.GetUnitOfWork();
@@ -1908,6 +1947,10 @@ namespace Umbraco.Core.Services
 
             using (new WriteLock(Locker))
             {
+                //Hack: this ensures that the entity's path is valid and if not it fixes/persists it
+                //see: http://issues.umbraco.org/issue/U4-9336
+                content.EnsureValidPath(Logger, entity => GetById(entity.ParentId), QuickUpdate);
+
                 var result = new List<Attempt<PublishStatus>>();
 
                 //Check if parent is published (although not if its a root node) - if parent isn't published this Content cannot be published

src/Umbraco.Core/Services/ContentTypeService.cs

@@ -141,7 +141,7 @@ namespace Umbraco.Core.Services
         {
             var evtMsgs = EventMessagesFactory.Get();
 
-            if (container.ContainedObjectType != containerObjectType)
+            if (container.ContainerObjectType != containerObjectType)
             {
                 var ex = new InvalidOperationException("Not a " + objectTypeName + " container.");
                 return OperationStatus.Exception(evtMsgs, ex);
@@ -799,7 +799,7 @@ namespace Umbraco.Core.Services
                     // of a different type, move them to the recycle bin, then permanently delete the content items.
                     // The main problem with this is that for every content item being deleted, events are raised...
                     // which we need for many things like keeping caches in sync, but we can surely do this MUCH better.
-                    
+
                     var deletedContentTypes = new List<IContentType>() {contentType};
                     deletedContentTypes.AddRange(contentType.Descendants().OfType<IContentType>());
 
@@ -807,7 +807,7 @@ namespace Umbraco.Core.Services
                     {
                         _contentService.DeleteContentOfType(deletedContentType.Id);
                     }
-                    
+
                     repository.Delete(contentType);
                     uow.Commit();
 

src/Umbraco.Core/Services/EntityService.cs

@@ -62,6 +62,12 @@ namespace Umbraco.Core.Services
 
         }
 
+        #region Static Queries
+
+        private IQuery<IUmbracoEntity> _rootEntityQuery;
+
+        #endregion
+
         /// <summary>
         /// Returns the integer id for a given GUID
         /// </summary>
@@ -386,11 +392,16 @@ namespace Umbraco.Core.Services
         /// <returns>An enumerable list of <see cref="IUmbracoEntity"/> objects</returns>
         public virtual IEnumerable<IUmbracoEntity> GetRootEntities(UmbracoObjectTypes umbracoObjectType)
         {
+            //create it once if it is needed (no need for locking here)
+            if (_rootEntityQuery == null)
+            {
+                _rootEntityQuery = Query<IUmbracoEntity>.Builder.Where(x => x.ParentId == -1);
+            }
+
             var objectTypeId = umbracoObjectType.GetGuid();
             using (var repository = RepositoryFactory.CreateEntityRepository(UowProvider.GetUnitOfWork()))
             {
-                var query = Query<IUmbracoEntity>.Builder.Where(x => x.ParentId == -1);
-                var entities = repository.GetByQuery(query, objectTypeId);
+                var entities = repository.GetByQuery(_rootEntityQuery, objectTypeId);
 
                 return entities;
             }

src/Umbraco.Core/Services/IMediaService.cs

@@ -1,6 +1,7 @@
 using System;
 using System.Collections.Generic;
 using System.ComponentModel;
+using System.Xml.Linq;
 using Umbraco.Core.Configuration;
 using Umbraco.Core.Models;
 using Umbraco.Core.Persistence.DatabaseModelDefinitions;
@@ -57,6 +58,19 @@ namespace Umbraco.Core.Services
     /// </summary>
     public interface IMediaService : IService
     {
+        /// <summary>
+        /// Gets all XML entries found in the cmsContentXml table based on the given path
+        /// </summary>
+        /// <param name="path">Path starts with</param>
+        /// <param name="pageIndex">Page number</param>
+        /// <param name="pageSize">Page size</param>
+        /// <param name="totalRecords">Total records the query would return without paging</param>
+        /// <returns>A paged enumerable of XML entries of media items</returns>
+        /// <remarks>
+        /// If -1 is passed, then this will return all media xml entries, otherwise will return all descendents from the path
+        /// </remarks>
+        IEnumerable<XElement> GetPagedXmlEntries(string path, long pageIndex, int pageSize, out long totalRecords);
+
         /// <summary>
         /// Rebuilds all xml content in the cmsContentXml table for all media
         /// </summary>

src/Umbraco.Core/Services/MediaService.cs

@@ -2,6 +2,7 @@ using System;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Globalization;
+using System.IO;
 using System.Linq;
 using System.Text.RegularExpressions;
 using System.Threading;
@@ -527,6 +528,10 @@ namespace Umbraco.Core.Services
         /// <returns>An Enumerable flat list of <see cref="IMedia"/> objects</returns>
         public IEnumerable<IMedia> GetDescendants(IMedia media)
         {
+            //This is a check to ensure that the path is correct for this entity to avoid problems like: http://issues.umbraco.org/issue/U4-9336 due to data corruption
+            if (media.ValidatePath() == false)
+                throw new InvalidDataException(string.Format("The content item {0} has an invalid path: {1} with parentID: {2}", media.Id, media.Path, media.ParentId));
+
             var uow = UowProvider.GetUnitOfWork();
             using (var repository = RepositoryFactory.CreateMediaRepository(uow))
             {
@@ -619,7 +624,7 @@ namespace Umbraco.Core.Services
         public IMedia GetMediaByPath(string mediaPath)
         {
             var umbracoFileValue = mediaPath;
-            
+
             const string Pattern = ".*[_][0-9]+[x][0-9]+[.].*";
             var isResized = Regex.IsMatch(mediaPath, Pattern);
 
@@ -998,56 +1003,63 @@ namespace Umbraco.Core.Services
         {
             if (media == null) throw new ArgumentNullException("media");
 
-            var originalPath = media.Path;
-
             var evtMsgs = EventMessagesFactory.Get();
 
-            if (Trashing.IsRaisedEventCancelled(
-                new MoveEventArgs<IMedia>(new MoveEventInfo<IMedia>(media, originalPath, Constants.System.RecycleBinMedia)), this))
+            using (new WriteLock(Locker))
             {
-                return OperationStatus.Cancelled(evtMsgs);
-            }
+                //Hack: this ensures that the entity's path is valid and if not it fixes/persists it
+                //see: http://issues.umbraco.org/issue/U4-9336
+                media.EnsureValidPath(Logger, entity => GetById(entity.ParentId), QuickUpdate);
 
-            var moveInfo = new List<MoveEventInfo<IMedia>>
-            {
-                new MoveEventInfo<IMedia>(media, originalPath, Constants.System.RecycleBinMedia)
-            };
+                var originalPath = media.Path;
 
-            //Find Descendants, which will be moved to the recycle bin along with the parent/grandparent.
-            var descendants = GetDescendants(media).OrderBy(x => x.Level).ToList();
-
-            var uow = UowProvider.GetUnitOfWork();
-            using (var repository = RepositoryFactory.CreateMediaRepository(uow))
-            {
-                //TODO: This should be part of the repo!
-
-                //Remove 'published' xml from the cmsContentXml table for the unpublished media
-                uow.Database.Delete<ContentXmlDto>("WHERE nodeId = @Id", new { Id = media.Id });
-
-                media.ChangeTrashedState(true, Constants.System.RecycleBinMedia);
-                repository.AddOrUpdate(media);
-
-                //Loop through descendants to update their trash state, but ensuring structure by keeping the ParentId
-                foreach (var descendant in descendants)
+                if (Trashing.IsRaisedEventCancelled(
+                    new MoveEventArgs<IMedia>(new MoveEventInfo<IMedia>(media, originalPath, Constants.System.RecycleBinMedia)), this))
                 {
-                    //Remove 'published' xml from the cmsContentXml table for the unpublished media
-                    uow.Database.Delete<ContentXmlDto>("WHERE nodeId = @Id", new { Id = descendant.Id });
-
-                    descendant.ChangeTrashedState(true, descendant.ParentId);
-                    repository.AddOrUpdate(descendant);
-
-                    moveInfo.Add(new MoveEventInfo<IMedia>(descendant, descendant.Path, descendant.ParentId));
+                    return OperationStatus.Cancelled(evtMsgs);
                 }
 
-                uow.Commit();
+                var moveInfo = new List<MoveEventInfo<IMedia>>
+                {
+                    new MoveEventInfo<IMedia>(media, originalPath, Constants.System.RecycleBinMedia)
+                };
+
+                //Find Descendants, which will be moved to the recycle bin along with the parent/grandparent.
+                var descendants = GetDescendants(media).OrderBy(x => x.Level).ToList();
+
+                var uow = UowProvider.GetUnitOfWork();
+                using (var repository = RepositoryFactory.CreateMediaRepository(uow))
+                {
+                    //TODO: This should be part of the repo!
+
+                    //Remove 'published' xml from the cmsContentXml table for the unpublished media
+                    uow.Database.Delete<ContentXmlDto>("WHERE nodeId = @Id", new { Id = media.Id });
+
+                    media.ChangeTrashedState(true, Constants.System.RecycleBinMedia);
+                    repository.AddOrUpdate(media);
+
+                    //Loop through descendants to update their trash state, but ensuring structure by keeping the ParentId
+                    foreach (var descendant in descendants)
+                    {
+                        //Remove 'published' xml from the cmsContentXml table for the unpublished media
+                        uow.Database.Delete<ContentXmlDto>("WHERE nodeId = @Id", new { Id = descendant.Id });
+
+                        descendant.ChangeTrashedState(true, descendant.ParentId);
+                        repository.AddOrUpdate(descendant);
+
+                        moveInfo.Add(new MoveEventInfo<IMedia>(descendant, descendant.Path, descendant.ParentId));
+                    }
+
+                    uow.Commit();
+                }
+
+                Trashed.RaiseEvent(
+                    new MoveEventArgs<IMedia>(false, evtMsgs, moveInfo.ToArray()), this);
+
+                Audit(AuditType.Move, "Move Media to Recycle Bin performed by user", userId, media.Id);
+
+                return OperationStatus.Success(evtMsgs);
             }
-
-            Trashed.RaiseEvent(
-                new MoveEventArgs<IMedia>(false, evtMsgs, moveInfo.ToArray()), this);
-
-            Audit(AuditType.Move, "Move Media to Recycle Bin performed by user", userId, media.Id);
-
-            return OperationStatus.Success(evtMsgs);
         }
 
         /// <summary>
@@ -1064,8 +1076,6 @@ namespace Umbraco.Core.Services
             ((IMediaServiceOperations)this).Delete(media, userId);
         }
 
-
-
         /// <summary>
         /// Permanently deletes versions from an <see cref="IMedia"/> object prior to a specific date.
         /// This method will never delete the latest version of a content item.
@@ -1154,7 +1164,6 @@ namespace Umbraco.Core.Services
         public bool Sort(IEnumerable<IMedia> items, int userId = 0, bool raiseEvents = true)
         {
             var asArray = items.ToArray();
-
             if (raiseEvents)
             {
                 if (Saving.IsRaisedEventCancelled(new SaveEventArgs<IMedia>(asArray), this))
@@ -1199,6 +1208,27 @@ namespace Umbraco.Core.Services
             return true;
         }
 
+        /// <summary>
+        /// Gets paged media descendants as XML by path
+        /// </summary>
+        /// <param name="path">Path starts with</param>
+        /// <param name="pageIndex">Page number</param>
+        /// <param name="pageSize">Page size</param>
+        /// <param name="totalRecords">Total records the query would return without paging</param>
+        /// <returns>A paged enumerable of XML entries of media items</returns>
+        public IEnumerable<XElement> GetPagedXmlEntries(string path, long pageIndex, int pageSize, out long totalRecords)
+        {
+            Mandate.ParameterCondition(pageIndex >= 0, "pageIndex");
+            Mandate.ParameterCondition(pageSize > 0, "pageSize");
+
+            var uow = UowProvider.GetUnitOfWork();
+            using (var repository = RepositoryFactory.CreateMediaRepository(uow))
+            {
+                var contents = repository.GetPagedXmlEntriesByPath(path, pageIndex, pageSize, out totalRecords);
+                return contents;
+            }
+        }
+
         /// <summary>
         /// Rebuilds all xml content in the cmsContentXml table for all media
         /// </summary>
@@ -1295,6 +1325,23 @@ namespace Umbraco.Core.Services
             }
         }
 
+        /// <summary>
+        /// Hack: This is used to fix some data if an entity's properties are invalid/corrupt
+        /// </summary>
+        /// <param name="media"></param>
+        private void QuickUpdate(IMedia media)
+        {
+            if (media == null) throw new ArgumentNullException("media");
+            if (media.HasIdentity == false) throw new InvalidOperationException("Cannot update an entity without an Identity");
+
+            var uow = UowProvider.GetUnitOfWork();
+            using (var repository = RepositoryFactory.CreateMediaRepository(uow))
+            {
+                repository.AddOrUpdate(media);
+                uow.Commit();
+            }
+        }
+
         #region Event Handlers
 
         /// <summary>

src/Umbraco.Core/StringExtensions.cs

@@ -184,7 +184,7 @@ namespace Umbraco.Core
         /// <param name="input"></param>
         /// <param name="ignoreFromClean"></param>
         /// <returns></returns>
-        internal static string CleanForXss(this string input, params char[] ignoreFromClean)
+        public static string CleanForXss(this string input, params char[] ignoreFromClean)
         {
             //remove any html
             input = input.StripHtml();

src/Umbraco.Core/Sync/ApplicationUrlHelper.cs

@@ -114,7 +114,8 @@ namespace Umbraco.Core.Sync
             // - contain a scheme
             // - end or not with a slash, it will be taken care of
             // eg "http://www.mysite.com/umbraco"
-            var registrar = ServerRegistrarResolver.Current.Registrar as IServerRegistrar2;
+            var resolver = ServerRegistrarResolver.HasCurrent ? ServerRegistrarResolver.Current : null;
+            var registrar = resolver == null ? null : resolver.Registrar as IServerRegistrar2;
             url = registrar == null ? null : registrar.GetCurrentServerUmbracoApplicationUrl();
             if (url.IsNullOrWhiteSpace() == false)
             {

src/Umbraco.Core/Sync/ColdBootType.cs

@@ -0,0 +1,24 @@
+namespace Umbraco.Core.Sync
+{
+    /// <summary>
+    /// The type of Cold Boot taking place
+    /// </summary>
+    public enum ColdBootType
+    {
+        /// <summary>
+        /// Indicates there was no cold boot
+        /// </summary>
+        NoColdBoot,
+
+        /// <summary>
+        /// This indicates that there is no lasysynced file for the current machine/appid
+        /// </summary>
+        NeverSynced,
+
+        /// <summary>
+        /// This indicates that the number of instructions needing to be processed exceeds the value stored 
+        /// for the <see cref="DatabaseServerMessengerOptions.MaxProcessingInstructionCount"/>
+        /// </summary>
+        ExceedsMaxProcessingInstructionCount
+    }
+}

src/Umbraco.Core/Sync/DatabaseServerMessenger.cs

@@ -29,7 +29,6 @@ namespace Umbraco.Core.Sync
     public class DatabaseServerMessenger : ServerMessengerBase
     {
         private readonly ApplicationContext _appContext;
-        private readonly DatabaseServerMessengerOptions _options;
         private readonly ManualResetEvent _syncIdle;
         private readonly object _locko = new object();
         private readonly ILogger _logger;
@@ -41,8 +40,14 @@ namespace Umbraco.Core.Sync
         private bool _released;
         private readonly ProfilingLogger _profilingLogger;
 
+        protected DatabaseServerMessengerOptions Options { get; private set; }
         protected ApplicationContext ApplicationContext { get { return _appContext; } }
 
+        /// <summary>
+        /// Returns the cold boot type detected for the current app domain
+        /// </summary>
+        public ColdBootType ColdBootType { get; private set; }
+
         public DatabaseServerMessenger(ApplicationContext appContext, bool distributedEnabled, DatabaseServerMessengerOptions options)
             : base(distributedEnabled)
         {
@@ -50,7 +55,7 @@ namespace Umbraco.Core.Sync
             if (options == null) throw new ArgumentNullException("options");
 
             _appContext = appContext;
-            _options = options;
+            Options = options;
             _lastPruned = _lastSync = DateTime.UtcNow;
             _syncIdle = new ManualResetEvent(true);
             _profilingLogger = appContext.ProfilingLogger;
@@ -115,7 +120,17 @@ namespace Umbraco.Core.Sync
                     {
                         _released = true; // no more syncs
                     }
-                    _syncIdle.WaitOne(); // wait for pending sync
+
+                    // wait a max of 5 seconds and then return, so that we don't block
+                    // the entire MainDom callbacks chain and prevent the AppDomain from
+                    // properly releasing MainDom - a timeout here means that one refresher
+                    // is taking too much time processing, however when it's done we will
+                    // not update lastId and stop everything
+                    var idle =_syncIdle.WaitOne(5000);
+                    if (idle == false)
+                    {
+                        _logger.Warn<DatabaseServerMessenger>("The wait lock timed out, application is shutting down. The current instruction batch will be re-processed.");
+                    }
                 },
                 weight);
 
@@ -136,11 +151,12 @@ namespace Umbraco.Core.Sync
         /// </remarks>
         private void Initialize()
         {
+            ColdBootType = ColdBootType.NoColdBoot;
+
             lock (_locko)
             {
                 if (_released) return;
-
-                var coldboot = false;
+                
                 if (_lastId < 0) // never synced before
                 {
                     // we haven't synced - in this case we aren't going to sync the whole thing, we will assume this is a new
@@ -149,25 +165,27 @@ namespace Umbraco.Core.Sync
                         + " The server will build its caches and indexes, and then adjust its last synced Id to the latest found in"
                         + " the database and maintain cache updates based on that Id.");
 
-                    coldboot = true;
+                    ColdBootType = ColdBootType.NeverSynced;
                 }
                 else
                 {
                     //check for how many instructions there are to process
+                    //TODO: In 7.6 we need to store the count of instructions per row since this is not affective because there can be far more than one (if not thousands)
+                    // of instructions in a single row.
                     var count = _appContext.DatabaseContext.Database.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoCacheInstruction WHERE id > @lastId", new {lastId = _lastId});
-                    if (count > _options.MaxProcessingInstructionCount)
+                    if (count > Options.MaxProcessingInstructionCount)
                     {
                         //too many instructions, proceed to cold boot
                         _logger.Warn<DatabaseServerMessenger>("The instruction count ({0}) exceeds the specified MaxProcessingInstructionCount ({1})."
                             + " The server will skip existing instructions, rebuild its caches and indexes entirely, adjust its last synced Id"
                             + " to the latest found in the database and maintain cache updates based on that Id.",
-                            () => count, () => _options.MaxProcessingInstructionCount);
+                            () => count, () => Options.MaxProcessingInstructionCount);
 
-                        coldboot = true;
+                        ColdBootType = ColdBootType.ExceedsMaxProcessingInstructionCount;
                     }
                 }
 
-                if (coldboot)
+                if (ColdBootType != ColdBootType.NoColdBoot)
                 {
                     // go get the last id in the db and store it
                     // note: do it BEFORE initializing otherwise some instructions might get lost
@@ -179,8 +197,8 @@ namespace Umbraco.Core.Sync
                         SaveLastSynced(maxId);
 
                     // execute initializing callbacks
-                    if (_options.InitializingCallbacks != null)
-                        foreach (var callback in _options.InitializingCallbacks)
+                    if (Options.InitializingCallbacks != null)
+                        foreach (var callback in Options.InitializingCallbacks)
                             callback();
                 }
 
@@ -198,12 +216,14 @@ namespace Umbraco.Core.Sync
                 if (_syncing)
                     return;
 
+                //Don't continue if we are released
                 if (_released)
                     return;
 
-                if ((DateTime.UtcNow - _lastSync).TotalSeconds <= _options.ThrottleSeconds)
+                if ((DateTime.UtcNow - _lastSync).TotalSeconds <= Options.ThrottleSeconds)
                     return;
 
+                //Set our flag and the lock to be in it's original state (i.e. it can be awaited)
                 _syncing = true;
                 _syncIdle.Reset();
                 _lastSync = DateTime.UtcNow;
@@ -215,7 +235,8 @@ namespace Umbraco.Core.Sync
                 {
                     ProcessDatabaseInstructions();
 
-                    if ((DateTime.UtcNow - _lastPruned).TotalSeconds <= _options.PruneThrottleSeconds)
+                    //Check for pruning throttling
+                    if ((_released || (DateTime.UtcNow - _lastPruned).TotalSeconds <= Options.PruneThrottleSeconds))
                         return;
 
                     _lastPruned = _lastSync;
@@ -231,7 +252,12 @@ namespace Umbraco.Core.Sync
             }
             finally
             {
-                _syncing = false;
+                lock (_locko)
+                {
+                    //We must reset our flag and signal any waiting locks
+                    _syncing = false;
+                }
+
                 _syncIdle.Set();
             }
         }
@@ -255,13 +281,17 @@ namespace Umbraco.Core.Sync
             //
             // FIXME not true if we're running on a background thread, assuming we can?
 
+
             var sql = new Sql().Select("*")
                 .From<CacheInstructionDto>(_appContext.DatabaseContext.SqlSyntax)
                 .Where<CacheInstructionDto>(dto => dto.Id > _lastId)
                 .OrderBy<CacheInstructionDto>(dto => dto.Id, _appContext.DatabaseContext.SqlSyntax);
 
-            var dtos = _appContext.DatabaseContext.Database.Fetch<CacheInstructionDto>(sql);
-            if (dtos.Count <= 0) return;
+            //only retrieve the top 100 (just in case there's tons)
+            // even though MaxProcessingInstructionCount is by default 1000 we still don't want to process that many
+            // rows in one request thread since each row can contain a ton of instructions (until 7.5.5 in which case
+            // a row can only contain MaxProcessingInstructionCount)
+            var topSql = _appContext.DatabaseContext.SqlSyntax.SelectTop(sql, 100);
 
             // only process instructions coming from a remote server, and ignore instructions coming from
             // the local server as they've already been processed. We should NOT assume that the sequence of
@@ -269,8 +299,22 @@ namespace Umbraco.Core.Sync
             var localIdentity = LocalIdentity;
 
             var lastId = 0;
-            foreach (var dto in dtos)
+
+            //tracks which ones have already been processed to avoid duplicates
+            var processed = new HashSet<RefreshInstruction>();
+
+            //It would have been nice to do this in a Query instead of Fetch using a data reader to save
+            // some memory however we cannot do thta because inside of this loop the cache refreshers are also
+            // performing some lookups which cannot be done with an active reader open
+            foreach (var dto in _appContext.DatabaseContext.Database.Fetch<CacheInstructionDto>(topSql))
             {
+                //If this flag gets set it means we're shutting down! In this case, we need to exit asap and cannot
+                // continue processing anything otherwise we'll hold up the app domain shutdown
+                if (_released)
+                {
+                    break;
+                }
+
                 if (dto.OriginIdentity == localIdentity)
                 {
                     // just skip that local one but update lastId nevertheless
@@ -291,27 +335,69 @@ namespace Umbraco.Core.Sync
                     continue;
                 }
 
-                // execute remote instructions & update lastId
-                try
-                {
-                    NotifyRefreshers(jsonA);
-                    lastId = dto.Id;
-                }
-                catch (Exception ex)
-                {
-                    _logger.Error<DatabaseServerMessenger>(
-                        string.Format("DISTRIBUTED CACHE IS NOT UPDATED. Failed to execute instructions ({0}: \"{1}\"). Instruction is being skipped/ignored", dto.Id, dto.Instructions), ex);
+                var instructionBatch = GetAllInstructions(jsonA);
 
-                    //we cannot throw here because this invalid instruction will just keep getting processed over and over and errors
-                    // will be thrown over and over. The only thing we can do is ignore and move on.
-                    lastId = dto.Id;
+                //process as per-normal
+                var success = ProcessDatabaseInstructions(instructionBatch, dto, processed, ref lastId);
+
+                //if they couldn't be all processed (i.e. we're shutting down) then exit
+                if (success == false)
+                {
+                    _logger.Info<DatabaseServerMessenger>("The current batch of instructions was not processed, app is shutting down");
+                    break;
                 }
+
             }
 
             if (lastId > 0)
                 SaveLastSynced(lastId);
         }
 
+        /// <summary>
+        /// Processes the instruction batch and checks for errors
+        /// </summary>
+        /// <param name="instructionBatch"></param>
+        /// <param name="dto"></param>
+        /// <param name="processed">
+        /// Tracks which instructions have already been processed to avoid duplicates
+        /// </param>
+        /// <param name="lastId"></param>
+        /// <returns>
+        /// returns true if all instructions in the batch were processed, otherwise false if they could not be due to the app being shut down
+        /// </returns>
+        private bool ProcessDatabaseInstructions(IReadOnlyCollection<RefreshInstruction> instructionBatch, CacheInstructionDto dto, HashSet<RefreshInstruction> processed, ref int lastId)
+        {
+            // execute remote instructions & update lastId
+            try
+            {
+                var result = NotifyRefreshers(instructionBatch, processed);
+                if (result)
+                {
+                    //if all instructions we're processed, set the last id
+                    lastId = dto.Id;
+                }
+                return result;
+            }
+            //catch (ThreadAbortException ex)
+            //{
+            //    //This will occur if the instructions processing is taking too long since this is occuring on a request thread.
+            //    // Or possibly if IIS terminates the appdomain. In any case, we should deal with this differently perhaps...
+            //}
+            catch (Exception ex)
+            {
+                _logger.Error<DatabaseServerMessenger>(
+                    string.Format("DISTRIBUTED CACHE IS NOT UPDATED. Failed to execute instructions (id: {0}, instruction count: {1}). Instruction is being skipped/ignored", dto.Id, instructionBatch.Count), ex);
+
+                //we cannot throw here because this invalid instruction will just keep getting processed over and over and errors
+                // will be thrown over and over. The only thing we can do is ignore and move on.
+                lastId = dto.Id;
+                return false;
+            }
+
+            ////if this is returned it will not be saved
+            //return -1;
+        }
+
         /// <summary>
         /// Remove old instructions from the database
         /// </summary>
@@ -322,7 +408,7 @@ namespace Umbraco.Core.Sync
         /// </remarks>
         private void PruneOldInstructions()
         {
-            var pruneDate = DateTime.UtcNow.AddDays(-_options.DaysToRetainInstructions);
+            var pruneDate = DateTime.UtcNow.AddDays(-Options.DaysToRetainInstructions);
 
             // using 2 queries is faster than convoluted joins
 
@@ -459,8 +545,14 @@ namespace Umbraco.Core.Sync
             return jsonRefresher;
         }
 
-        private static void NotifyRefreshers(IEnumerable<JToken> jsonArray)
+        /// <summary>
+        /// Parses out the individual instructions to be processed
+        /// </summary>
+        /// <param name="jsonArray"></param>
+        /// <returns></returns>
+        private static List<RefreshInstruction> GetAllInstructions(IEnumerable<JToken> jsonArray)
         {
+            var result = new List<RefreshInstruction>();
             foreach (var jsonItem in jsonArray)
             {
                 // could be a JObject in which case we can convert to a RefreshInstruction,
@@ -469,35 +561,64 @@ namespace Umbraco.Core.Sync
                 if (jsonObj != null)
                 {
                     var instruction = jsonObj.ToObject<RefreshInstruction>();
-                    switch (instruction.RefreshType)
-                    {
-                        case RefreshMethodType.RefreshAll:
-                            RefreshAll(instruction.RefresherId);
-                            break;
-                        case RefreshMethodType.RefreshByGuid:
-                            RefreshByGuid(instruction.RefresherId, instruction.GuidId);
-                            break;
-                        case RefreshMethodType.RefreshById:
-                            RefreshById(instruction.RefresherId, instruction.IntId);
-                            break;
-                        case RefreshMethodType.RefreshByIds:
-                            RefreshByIds(instruction.RefresherId, instruction.JsonIds);
-                            break;
-                        case RefreshMethodType.RefreshByJson:
-                            RefreshByJson(instruction.RefresherId, instruction.JsonPayload);
-                            break;
-                        case RefreshMethodType.RemoveById:
-                            RemoveById(instruction.RefresherId, instruction.IntId);
-                            break;
-                    }
-
+                    result.Add(instruction);
                 }
                 else
                 {
-                    var jsonInnerArray = (JArray) jsonItem;
-                    NotifyRefreshers(jsonInnerArray); // recurse
+                    var jsonInnerArray = (JArray)jsonItem;
+                    result.AddRange(GetAllInstructions(jsonInnerArray)); // recurse
                 }
             }
+            return result;
+        }
+
+        /// <summary>
+        /// executes the instructions against the cache refresher instances
+        /// </summary>
+        /// <param name="instructions"></param>
+        /// <param name="processed"></param>
+        /// <returns>
+        /// Returns true if all instructions were processed, otherwise false if the processing was interupted (i.e. app shutdown)
+        /// </returns>
+        private bool NotifyRefreshers(IEnumerable<RefreshInstruction> instructions, HashSet<RefreshInstruction> processed)
+        {
+            foreach (var instruction in instructions)
+            {
+                //Check if the app is shutting down, we need to exit if this happens.
+                if (_released)
+                {
+                    return false;
+                }
+
+                //this has already been processed
+                if (processed.Contains(instruction))
+                    continue;
+
+                switch (instruction.RefreshType)
+                {
+                    case RefreshMethodType.RefreshAll:
+                        RefreshAll(instruction.RefresherId);
+                        break;
+                    case RefreshMethodType.RefreshByGuid:
+                        RefreshByGuid(instruction.RefresherId, instruction.GuidId);
+                        break;
+                    case RefreshMethodType.RefreshById:
+                        RefreshById(instruction.RefresherId, instruction.IntId);
+                        break;
+                    case RefreshMethodType.RefreshByIds:
+                        RefreshByIds(instruction.RefresherId, instruction.JsonIds);
+                        break;
+                    case RefreshMethodType.RefreshByJson:
+                        RefreshByJson(instruction.RefresherId, instruction.JsonPayload);
+                        break;
+                    case RefreshMethodType.RemoveById:
+                        RemoveById(instruction.RefresherId, instruction.IntId);
+                        break;
+                }
+
+                processed.Add(instruction);
+            }
+            return true;
         }
 
         private static void RefreshAll(Guid uniqueIdentifier)

src/Umbraco.Core/Sync/SingleServerRegistrar.cs

@@ -0,0 +1,37 @@
+using System.Collections.Generic;
+
+namespace Umbraco.Core.Sync
+{
+    public class SingleServerRegistrar : IServerRegistrar2
+    {
+        private readonly string _umbracoApplicationUrl;
+
+        public IEnumerable<IServerAddress> Registrations { get; private set; }
+
+        public SingleServerRegistrar()
+        {
+            _umbracoApplicationUrl = ApplicationContext.Current.UmbracoApplicationUrl;
+            Registrations = new[] { new ServerAddressImpl(_umbracoApplicationUrl) };
+        }
+
+        public ServerRole GetCurrentServerRole()
+        {
+            return ServerRole.Single;
+        }
+
+        public string GetCurrentServerUmbracoApplicationUrl()
+        {
+            return _umbracoApplicationUrl;
+        }
+
+        private class ServerAddressImpl : IServerAddress
+        {
+            public ServerAddressImpl(string serverAddress)
+            {
+                ServerAddress = serverAddress;
+            }
+
+            public string ServerAddress { get; private set; }
+        }
+    }
+}

src/Umbraco.Core/Umbraco.Core.csproj

@@ -21,7 +21,7 @@
     <DebugType>full</DebugType>
     <Optimize>false</Optimize>
     <OutputPath>bin\Debug\</OutputPath>
-    <DefineConstants>DEBUG;TRACE</DefineConstants>
+    <DefineConstants>TRACE;DEBUG</DefineConstants>
     <ErrorReport>prompt</ErrorReport>
     <WarningLevel>4</WarningLevel>
     <Prefer32Bit>false</Prefer32Bit>
@@ -39,67 +39,54 @@
   <ItemGroup>
     <Reference Include="AutoMapper, Version=3.3.1.0, Culture=neutral, PublicKeyToken=be96cd2c38ef1005, processorArchitecture=MSIL">
       <HintPath>..\packages\AutoMapper.3.3.1\lib\net40\AutoMapper.dll</HintPath>
-      <Private>True</Private>
     </Reference>
     <Reference Include="AutoMapper.Net4, Version=3.3.1.0, Culture=neutral, PublicKeyToken=be96cd2c38ef1005, processorArchitecture=MSIL">
       <HintPath>..\packages\AutoMapper.3.3.1\lib\net40\AutoMapper.Net4.dll</HintPath>
-      <Private>True</Private>
     </Reference>
     <Reference Include="HtmlAgilityPack, Version=1.4.9.0, Culture=neutral, PublicKeyToken=bd319b19eaf3b43a, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\HtmlAgilityPack.1.4.9\lib\Net45\HtmlAgilityPack.dll</HintPath>
     </Reference>
-    <Reference Include="ICSharpCode.SharpZipLib">
+    <Reference Include="ICSharpCode.SharpZipLib, Version=0.86.0.518, Culture=neutral, PublicKeyToken=1b03e6acf1164f73, processorArchitecture=MSIL">
       <HintPath>..\packages\SharpZipLib.0.86.0\lib\20\ICSharpCode.SharpZipLib.dll</HintPath>
     </Reference>
-    <Reference Include="ImageProcessor, Version=2.4.5.0, Culture=neutral, processorArchitecture=MSIL">
-      <HintPath>..\packages\ImageProcessor.2.4.5.0\lib\net45\ImageProcessor.dll</HintPath>
-      <Private>True</Private>
+    <Reference Include="ImageProcessor, Version=2.5.1.0, Culture=neutral, processorArchitecture=MSIL">
+      <HintPath>..\packages\ImageProcessor.2.5.1\lib\net45\ImageProcessor.dll</HintPath>
     </Reference>
     <Reference Include="log4net, Version=1.2.11.0, Culture=neutral, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\log4net-mediumtrust.2.0.0\lib\log4net.dll</HintPath>
     </Reference>
     <Reference Include="Microsoft.AspNet.Identity.Core, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\Microsoft.AspNet.Identity.Core.2.2.1\lib\net45\Microsoft.AspNet.Identity.Core.dll</HintPath>
     </Reference>
     <Reference Include="Microsoft.AspNet.Identity.Owin, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\Microsoft.AspNet.Identity.Owin.2.2.1\lib\net45\Microsoft.AspNet.Identity.Owin.dll</HintPath>
     </Reference>
     <Reference Include="Microsoft.Owin, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\Microsoft.Owin.3.0.1\lib\net45\Microsoft.Owin.dll</HintPath>
     </Reference>
-    <Reference Include="Microsoft.Owin.Security">
+    <Reference Include="Microsoft.Owin.Security, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
       <HintPath>..\packages\Microsoft.Owin.Security.3.0.1\lib\net45\Microsoft.Owin.Security.dll</HintPath>
-      <Private>True</Private>
     </Reference>
     <Reference Include="Microsoft.Owin.Security.Cookies, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\Microsoft.Owin.Security.Cookies.3.0.1\lib\net45\Microsoft.Owin.Security.Cookies.dll</HintPath>
     </Reference>
     <Reference Include="Microsoft.Owin.Security.OAuth, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\Microsoft.Owin.Security.OAuth.3.0.1\lib\net45\Microsoft.Owin.Security.OAuth.dll</HintPath>
     </Reference>
     <Reference Include="MiniProfiler, Version=2.1.0.0, Culture=neutral, PublicKeyToken=b44f9351044011a3, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\MiniProfiler.2.1.0\lib\net40\MiniProfiler.dll</HintPath>
     </Reference>
-    <Reference Include="MySql.Data">
-      <SpecificVersion>False</SpecificVersion>
+    <Reference Include="MySql.Data, Version=6.9.8.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d, processorArchitecture=MSIL">
       <HintPath>..\packages\MySql.Data.6.9.8\lib\net45\MySql.Data.dll</HintPath>
     </Reference>
     <Reference Include="Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
       <HintPath>..\packages\Newtonsoft.Json.6.0.8\lib\net45\Newtonsoft.Json.dll</HintPath>
       <Private>True</Private>
     </Reference>
-    <Reference Include="Owin">
+    <Reference Include="Owin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f0ebd12fd5e55cc5, processorArchitecture=MSIL">
       <HintPath>..\packages\Owin.1.0\lib\net40\Owin.dll</HintPath>
     </Reference>
-    <Reference Include="Semver">
+    <Reference Include="Semver, Version=1.1.2.0, Culture=neutral, processorArchitecture=MSIL">
       <HintPath>..\packages\semver.1.1.2\lib\net45\Semver.dll</HintPath>
     </Reference>
     <Reference Include="System" />
@@ -107,11 +94,9 @@
     <Reference Include="System.Configuration" />
     <Reference Include="System.Data.Entity" />
     <Reference Include="System.Data.SqlServerCe, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.dll</HintPath>
     </Reference>
     <Reference Include="System.Data.SqlServerCe.Entity, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
-      <SpecificVersion>False</SpecificVersion>
       <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.Entity.dll</HintPath>
     </Reference>
     <Reference Include="System.DirectoryServices.AccountManagement" />
@@ -296,6 +281,7 @@
     <Compile Include="Configuration\UmbracoSettings\ServerCollection.cs" />
     <Compile Include="Configuration\UmbracoSettings\ServerElement.cs" />
     <Compile Include="Configuration\UmbracoSettings\TemplatesElement.cs" />
+    <Compile Include="Configuration\UmbracoSettings\UmbracoConfigurationElement.cs" />
     <Compile Include="Configuration\UmbracoSettings\UmbracoSettingsSection.cs" />
     <Compile Include="Configuration\UmbracoSettings\UrlReplacingElement.cs" />
     <Compile Include="Configuration\UmbracoSettings\UserProviderElement.cs" />
@@ -415,6 +401,7 @@
     <Compile Include="Models\UmbracoDomain.cs" />
     <Compile Include="Models\DoNotCloneAttribute.cs" />
     <Compile Include="Models\IDomain.cs" />
+    <Compile Include="Persistence\BulkDataReader.cs" />
     <Compile Include="Persistence\DatabaseNodeLockExtensions.cs" />
     <Compile Include="Persistence\Factories\ExternalLoginFactory.cs" />
     <Compile Include="Persistence\Factories\MigrationEntryFactory.cs" />
@@ -428,6 +415,7 @@
     <Compile Include="Persistence\Mappers\MigrationEntryMapper.cs" />
     <Compile Include="Persistence\Migrations\LocalMigrationContext.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionFourOneZero\AddPreviewXmlTable.cs" />
+    <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFiveFive\UpdateAllowedMediaTypesAtRoot.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFiveZero\EnsureServersLockObject.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFiveZero\AddRedirectUrlTable.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFiveZero\RemoveStylesheetDataAndTablesAgain.cs" />
@@ -438,6 +426,7 @@
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFourZero\AddUniqueIdPropertyTypeGroupColumn.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFourZero\RemoveParentIdPropertyTypeGroupColumn.cs" />
     <Compile Include="Persistence\Mappers\TaskTypeMapper.cs" />
+    <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenSixZero\RemovePropertyDataIdIndex.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenThreeTwo\EnsureMigrationsTableIdentityIsCorrect.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenFourZero\EnsureContentTypeUniqueIdsAreConsistent.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenThreeZero\AddExternalLoginsTable.cs" />
@@ -472,6 +461,8 @@
     <Compile Include="Media\Exif\TIFFStrip.cs" />
     <Compile Include="Media\Exif\Utility.cs" />
     <Compile Include="Persistence\Migrations\Upgrades\TargetVersionSevenThreeOne\UpdateUserLanguagesToIsoCode.cs" />
+    <Compile Include="Persistence\PocoDataDataReader.cs" />
+    <Compile Include="Persistence\Querying\QueryExtensions.cs" />
     <Compile Include="Persistence\RecordPersistenceType.cs" />
     <Compile Include="Persistence\Relators\AccessRulesRelator.cs" />
     <Compile Include="Persistence\Repositories\AuditRepository.cs" />
@@ -1013,10 +1004,10 @@
     <Compile Include="Persistence\PetaPocoConnectionExtensions.cs" />
     <Compile Include="Persistence\PetaPocoExtensions.cs" />
     <Compile Include="Persistence\PetaPocoSqlExtensions.cs" />
-    <Compile Include="Persistence\Querying\BaseExpressionHelper.cs" />
-    <Compile Include="Persistence\Querying\PocoToSqlExpressionHelper.cs" />
+    <Compile Include="Persistence\Querying\ExpressionVisitorBase.cs" />
+    <Compile Include="Persistence\Querying\PocoToSqlExpressionVisitor.cs" />
     <Compile Include="Persistence\Querying\IQuery.cs" />
-    <Compile Include="Persistence\Querying\ModelToSqlExpressionHelper.cs" />
+    <Compile Include="Persistence\Querying\ModelToSqlExpressionVisitor.cs" />
     <Compile Include="Persistence\Querying\Query.cs" />
     <Compile Include="Persistence\Querying\SqlTranslator.cs" />
     <Compile Include="Persistence\Relators\DictionaryLanguageTextRelator.cs" />
@@ -1198,7 +1189,7 @@
     <Compile Include="Models\Rdbms\UserDto.cs" />
     <Compile Include="Models\Rdbms\UserTypeDto.cs" />
     <Compile Include="NameValueCollectionExtensions.cs" />
-    <Compile Include="ObjectResolution\WeightedPluginAttribute.cs" />
+    <Compile Include="ObjectResolution\WeightAttribute.cs" />
     <Compile Include="Persistence\PetaPoco.cs" />
     <Compile Include="PropertyEditors\ValueConverters\DatePickerValueConverter.cs" />
     <Compile Include="PropertyEditors\IPropertyEditorValueConverter.cs" />
@@ -1320,6 +1311,7 @@
     <Compile Include="Strings\ContentBaseExtensions.cs" />
     <Compile Include="Strings\Diff.cs" />
     <Compile Include="Sync\BatchedWebServiceServerMessenger.cs" />
+    <Compile Include="Sync\ColdBootType.cs" />
     <Compile Include="Sync\IServerRegistrar2.cs" />
     <Compile Include="Sync\ServerRole.cs" />
     <Compile Include="Sync\DatabaseServerMessenger.cs" />
@@ -1330,6 +1322,7 @@
     <Compile Include="Sync\ApplicationUrlHelper.cs" />
     <Compile Include="Sync\RefreshMethodType.cs" />
     <Compile Include="Sync\ServerMessengerBase.cs" />
+    <Compile Include="Sync\SingleServerRegistrar.cs" />
     <Compile Include="ThreadExtensions.cs" />
     <Compile Include="TopologicalSorter.cs" />
     <Compile Include="Strings\DefaultUrlSegmentProvider.cs" />
@@ -1434,9 +1427,7 @@
   <ItemGroup>
     <None Include="app.config" />
     <None Include="FileResources\BlockingWeb.config" />
-    <None Include="packages.config">
-      <SubType>Designer</SubType>
-    </None>
+    <None Include="packages.config" />
   </ItemGroup>
   <ItemGroup>
     <ProjectReference Include="..\umbraco.interfaces\umbraco.interfaces.csproj">

src/Umbraco.Core/packages.config

@@ -2,8 +2,8 @@
 <packages>
   <package id="AutoMapper" version="3.3.1" targetFramework="net45" />
   <package id="HtmlAgilityPack" version="1.4.9" targetFramework="net45" />
-  <package id="ImageProcessor" version="2.4.5.0" targetFramework="net45" />
-  <package id="log4net-mediumtrust" version="2.0.0" targetFramework="net4" />
+  <package id="ImageProcessor" version="2.5.1" targetFramework="net45" />
+  <package id="log4net-mediumtrust" version="2.0.0" targetFramework="net45" />
   <package id="Microsoft.AspNet.Identity.Core" version="2.2.1" targetFramework="net45" />
   <package id="Microsoft.AspNet.Identity.Owin" version="2.2.1" targetFramework="net45" />
   <package id="Microsoft.Owin" version="3.0.1" targetFramework="net45" />
@@ -15,6 +15,6 @@
   <package id="Newtonsoft.Json" version="6.0.8" targetFramework="net45" />
   <package id="Owin" version="1.0" targetFramework="net45" />
   <package id="semver" version="1.1.2" targetFramework="net45" />
-  <package id="SharpZipLib" version="0.86.0" targetFramework="net4" />
+  <package id="SharpZipLib" version="0.86.0" targetFramework="net45" />
   <package id="SqlServerCE" version="4.0.0.1" targetFramework="net45" />
 </packages>

src/Umbraco.Tests.Benchmarks/App.config

@@ -0,0 +1,48 @@
+<?xml version="1.0" encoding="utf-8"?>
+<configuration>
+    <startup> 
+        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
+    </startup>
+  <runtime>
+    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
+
+      <dependentAssembly>
+        <assemblyIdentity name="System.Data.SqlServerCe" publicKeyToken="89845dcd8080cc91" />
+        <bindingRedirect oldVersion="0.0.0.0-4.0.0.1" newVersion="4.0.0.1" />
+      </dependentAssembly>
+      
+      <dependentAssembly>
+        <assemblyIdentity name="Microsoft.Owin" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="Microsoft.Owin.Security.OAuth" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="Microsoft.Owin.Security" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="Microsoft.Owin.Security.Cookies" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="System.Web.Mvc" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-5.2.3.0" newVersion="5.2.3.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="System.Web.WebPages" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-3.0.0.0" newVersion="3.0.0.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="System.Web.Http" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-5.2.3.0" newVersion="5.2.3.0" />
+      </dependentAssembly>
+      <dependentAssembly>
+        <assemblyIdentity name="System.Net.Http.Formatting" publicKeyToken="31bf3856ad364e35" culture="neutral" />
+        <bindingRedirect oldVersion="0.0.0.0-5.2.3.0" newVersion="5.2.3.0" />
+      </dependentAssembly>
+    </assemblyBinding>
+  </runtime>
+</configuration>

src/Umbraco.Tests.Benchmarks/BulkInsertBenchmarks.cs

@@ -0,0 +1,202 @@
+using System;
+using System.Collections.Generic;
+using System.Data.SqlServerCe;
+using System.IO;
+using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Configs;
+using BenchmarkDotNet.Diagnostics.Windows;
+using BenchmarkDotNet.Jobs;
+using Umbraco.Core;
+using Umbraco.Core.Logging;
+using Umbraco.Core.Models.Rdbms;
+using Umbraco.Core.Persistence;
+using Umbraco.Core.Persistence.Migrations.Initial;
+using Umbraco.Core.Persistence.SqlSyntax;
+using Umbraco.Tests.TestHelpers;
+using ILogger = Umbraco.Core.Logging.ILogger;
+
+namespace Umbraco.Tests.Benchmarks
+{
+    [Config(typeof(Config))]
+    public class BulkInsertBenchmarks
+    {
+        private class Config : ManualConfig
+        {
+            public Config()
+            {
+                Add(new MemoryDiagnoser());
+                //Add(ExecutionValidator.FailOnError);
+
+                //The 'quick and dirty' settings, so it runs a little quicker
+                // see benchmarkdotnet FAQ
+                Add(Job.Default
+                    .WithLaunchCount(1) // benchmark process will be launched only once
+                    .WithIterationTime(100) // 100ms per iteration
+                    .WithWarmupCount(3) // 3 warmup iteration
+                    .WithTargetCount(3)); // 3 target iteration                
+            }
+        }
+
+        private static byte[] _initDbBytes = null;
+
+        [Setup]
+        public void Setup()
+        {
+            var logger = new DebugDiagnosticsLogger();
+            var path = TestHelper.CurrentAssemblyDirectory;
+
+            _sqlCeSyntax = new SqlCeSyntaxProvider();
+            _sqlServerSyntax = new SqlServerSyntaxProvider();
+
+            SetupSqlCe(path, logger);
+            SetupSqlServer(logger);
+
+            
+        }        
+
+        private void SetupSqlServer(ILogger logger)
+        {
+            //create the db
+            _dbSqlServer = new UmbracoDatabase(
+                "server=.\\SQLExpress;database=YOURDB;user id=YOURUSER;password=YOURPASS",
+                Constants.DatabaseProviders.SqlServer,
+                logger);
+
+            //drop the table
+            // note: DROP TABLE IF EXISTS is SQL 2016+
+            _dbSqlServer.Execute("IF OBJECT_ID('dbo.umbracoServer', 'U') IS NOT NULL DROP TABLE [umbracoServer]");
+
+            //re-create it
+            _dbSqlServer.Execute(@"CREATE TABLE [umbracoServer](
+	[id] [int] IDENTITY(1,1) NOT NULL,
+	[address] [nvarchar](500) NOT NULL,
+	[computerName] [nvarchar](255) NOT NULL,
+	[registeredDate] [datetime] NOT NULL CONSTRAINT [DF_umbracoServer_registeredDate]  DEFAULT (getdate()),
+	[lastNotifiedDate] [datetime] NOT NULL,
+	[isActive] [bit] NOT NULL,
+	[isMaster] [bit] NOT NULL,
+ CONSTRAINT [PK_umbracoServer] PRIMARY KEY CLUSTERED 
+(
+	[id] ASC
+)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
+)");
+        }
+
+        private void SetupSqlCe(string path, ILogger logger)
+        {
+            var dbName = string.Concat("Umb", Guid.NewGuid(), ".sdf");
+            AppDomain.CurrentDomain.SetData("DataDirectory", path);
+            var sqlCeConnectionString = $"Datasource=|DataDirectory|\\{dbName};Flush Interval=1;";
+            
+            _dbFile = Path.Combine(path, dbName);
+
+            //only create the db one time
+            if (_initDbBytes == null)
+            {
+                using (var engine = new SqlCeEngine(sqlCeConnectionString))
+                {
+                    engine.CreateDatabase();
+                }
+
+                //use the db  to create the initial schema so we can reuse in each bench
+                using (_dbSqlCe = new UmbracoDatabase(
+                    sqlCeConnectionString,
+                    Constants.DatabaseProviders.SqlCe,
+                    logger))
+                {
+                    var creation = new DatabaseSchemaCreation(_dbSqlCe, logger, _sqlCeSyntax);
+                    creation.InitializeDatabaseSchema();
+                }
+                _initDbBytes = File.ReadAllBytes(_dbFile);            
+            }
+            else
+            {
+                File.WriteAllBytes(_dbFile, _initDbBytes);
+            }            
+
+            //create the db
+            _dbSqlCe = new UmbracoDatabase(
+                sqlCeConnectionString,
+                Constants.DatabaseProviders.SqlCe,
+                logger);
+        }
+
+        private List<ServerRegistrationDto> GetData()
+        {
+            var data = new List<ServerRegistrationDto>();
+            for (var i = 0; i < 1000; i++)
+            {
+                data.Add(new ServerRegistrationDto
+                {
+                    ServerAddress = "address" + Guid.NewGuid(),
+                    ServerIdentity = "computer" + Guid.NewGuid(),
+                    DateRegistered = DateTime.Now,
+                    IsActive = true,
+                    DateAccessed = DateTime.Now
+                });
+            }
+            return data;
+        }
+
+        [Cleanup]
+        public void Cleanup()
+        {
+            _dbSqlCe.Dispose();
+            _dbSqlServer.Dispose();
+            File.Delete(_dbFile);
+        }
+
+        private string _dbFile;
+        private UmbracoDatabase _dbSqlCe;
+        private UmbracoDatabase _dbSqlServer;
+        private ISqlSyntaxProvider _sqlCeSyntax;
+        private ISqlSyntaxProvider _sqlServerSyntax;
+
+        /// <summary>
+        /// Tests updating the existing XML way
+        /// </summary>
+        [Benchmark(Baseline = true)]
+        public void SqlCeOneByOne()
+        {
+            using (var tr = _dbSqlCe.GetTransaction())
+            {
+                _dbSqlCe.BulkInsertRecords(GetData(), tr, _sqlCeSyntax, useNativeSqlPlatformBulkInsert: false);
+                tr.Complete();
+            }
+        }
+
+        /// <summary>
+        /// Tests updating with only the object graph
+        /// </summary>
+        [Benchmark]
+        public void SqlCeTableDirect()
+        {
+            using (var tr = _dbSqlCe.GetTransaction())
+            {
+                _dbSqlCe.BulkInsertRecords(GetData(), tr, _sqlCeSyntax, useNativeSqlPlatformBulkInsert: true);
+                tr.Complete();
+            }
+        }
+
+        [Benchmark]
+        public void SqlServerBulkInsertStatements()
+        {
+            using (var tr = _dbSqlServer.GetTransaction())
+            {
+                _dbSqlServer.BulkInsertRecords(GetData(), tr, _sqlServerSyntax, useNativeSqlPlatformBulkInsert: false);
+                tr.Complete();
+            }
+        }
+
+        [Benchmark]
+        public void SqlServerBulkCopy()
+        {
+            using (var tr = _dbSqlServer.GetTransaction())
+            {
+                _dbSqlServer.BulkInsertRecords(GetData(), tr, _sqlServerSyntax, useNativeSqlPlatformBulkInsert: true);
+                tr.Complete();
+            }
+        }
+
+    }
+}

src/Umbraco.Tests.Benchmarks/LinqCastBenchmarks.cs

@@ -0,0 +1,58 @@
+using System.Collections.Generic;
+using System.Linq;
+using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Configs;
+using BenchmarkDotNet.Diagnostics.Windows;
+
+namespace Umbraco.Tests.Benchmarks
+{
+    /// <summary>
+    /// Want to check what is faster OfType or Cast when a enurable all has the same items
+    /// </summary>
+    [Config(typeof(Config))]
+    public class LinqCastBenchmarks
+    {
+        private class Config : ManualConfig
+        {
+            public Config()
+            {
+                Add(new MemoryDiagnoser());
+            }
+        }
+
+        public LinqCastBenchmarks()
+        {
+            _array = new List<object>();
+            _array.AddRange(Enumerable.Range(0, 10000).Select(x => x.ToString()));
+        }
+
+        private readonly List<object> _array;
+
+        [Benchmark(Baseline = true)]
+        public void OfType()
+        {
+            foreach (var i in _array.OfType<string>())
+            {
+                var a = i;
+            }
+        }
+
+        [Benchmark()]
+        public void Cast()
+        {
+            foreach (var i in _array.Cast<string>())
+            {
+                var a = i;
+            }
+        }
+
+        [Benchmark()]
+        public void ExplicitCast()
+        {
+            foreach (var i in _array)
+            {
+                var a = (string)i;
+            }
+        }
+    }
+}

src/Umbraco.Tests.Benchmarks/ModelToSqlExpressionHelperBenchmarks.cs

@@ -0,0 +1,93 @@
+using System;
+using System.Collections.Generic;
+using System.Data.SqlServerCe;
+using System.Diagnostics;
+using System.IO;
+using System.Linq.Expressions;
+using System.Threading;
+using System.Xml;
+using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Columns;
+using BenchmarkDotNet.Configs;
+using BenchmarkDotNet.Diagnosers;
+using BenchmarkDotNet.Diagnostics.Windows;
+using BenchmarkDotNet.Jobs;
+using BenchmarkDotNet.Loggers;
+using BenchmarkDotNet.Reports;
+using BenchmarkDotNet.Running;
+using BenchmarkDotNet.Validators;
+using Umbraco.Core;
+using Umbraco.Core.Logging;
+using Umbraco.Core.Models;
+using Umbraco.Core.Models.Rdbms;
+using Umbraco.Core.Persistence;
+using Umbraco.Core.Persistence.Mappers;
+using Umbraco.Core.Persistence.Migrations.Initial;
+using Umbraco.Core.Persistence.Querying;
+using Umbraco.Core.Persistence.SqlSyntax;
+using Umbraco.Tests.TestHelpers;
+using ILogger = Umbraco.Core.Logging.ILogger;
+
+namespace Umbraco.Tests.Benchmarks
+{
+    [Config(typeof(Config))]
+    public class ModelToSqlExpressionHelperBenchmarks
+    {
+        private class Config : ManualConfig
+        {
+            public Config()
+            {
+                Add(new MemoryDiagnoser());                
+            }
+        }
+
+        public ModelToSqlExpressionHelperBenchmarks()
+        {
+            _contentMapper = new ContentMapper(_syntaxProvider);
+            _contentMapper.BuildMap();
+            _cachedExpression = new CachedExpression();
+        }
+        
+        private readonly ISqlSyntaxProvider _syntaxProvider = new SqlCeSyntaxProvider();
+        private readonly BaseMapper _contentMapper;
+        private readonly CachedExpression _cachedExpression;
+        
+        [Benchmark(Baseline = true)]
+        public void WithNonCached()
+        {
+            for (int i = 0; i < 100; i++)
+            {
+                var a = i;
+                var b = i*10;
+                Expression<Func<IContent, bool>> predicate = content =>
+                content.Path.StartsWith("-1") && content.Published && (content.ContentTypeId == a || content.ContentTypeId == b);
+
+                var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IContent>(_syntaxProvider, _contentMapper);
+                var result = modelToSqlExpressionHelper.Visit(predicate);
+            }
+            
+        }
+
+        
+
+        [Benchmark()]
+        public void WithCachedExpression()
+        {
+            for (int i = 0; i < 100; i++)
+            {
+                var a = i;
+                var b = i * 10;
+                Expression<Func<IContent, bool>> predicate = content =>
+                content.Path.StartsWith("-1") && content.Published && (content.ContentTypeId == a || content.ContentTypeId == b);
+                
+                var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IContent>(_syntaxProvider, _contentMapper);
+
+                //wrap it!
+                _cachedExpression.Wrap(predicate);
+
+                var result = modelToSqlExpressionHelper.Visit(_cachedExpression);
+            }
+        }
+
+    }
+}

src/Umbraco.Tests.Benchmarks/Program.cs

@@ -0,0 +1,21 @@
+using System;
+using System.Reflection;
+using BenchmarkDotNet.Running;
+
+namespace Umbraco.Tests.Benchmarks
+{
+    internal class Program
+    {
+        public static void Main(string[] args)
+        {
+            var switcher = new BenchmarkSwitcher(new[]
+            {
+                typeof(BulkInsertBenchmarks),
+                typeof(ModelToSqlExpressionHelperBenchmarks),
+                typeof(XmlBenchmarks),
+                typeof(LinqCastBenchmarks)
+            });
+            switcher.Run(args);            
+        }
+    }
+}

src/Umbraco.Tests.Benchmarks/Properties/AssemblyInfo.cs

@@ -0,0 +1,36 @@
+using System.Reflection;
+using System.Runtime.CompilerServices;
+using System.Runtime.InteropServices;
+
+// General Information about an assembly is controlled through the following 
+// set of attributes. Change these attribute values to modify the information
+// associated with an assembly.
+[assembly: AssemblyTitle("Umbraco.Tests.Benchmarks")]
+[assembly: AssemblyDescription("")]
+[assembly: AssemblyConfiguration("")]
+[assembly: AssemblyCompany("")]
+[assembly: AssemblyProduct("Umbraco.Tests.Benchmarks")]
+[assembly: AssemblyCopyright("Copyright ©  2016")]
+[assembly: AssemblyTrademark("")]
+[assembly: AssemblyCulture("")]
+
+// Setting ComVisible to false makes the types in this assembly not visible 
+// to COM components.  If you need to access a type in this assembly from 
+// COM, set the ComVisible attribute to true on that type.
+[assembly: ComVisible(false)]
+
+// The following GUID is for the ID of the typelib if this project is exposed to COM
+[assembly: Guid("86deb346-089f-4106-89c8-d852b9cf2a33")]
+
+// Version information for an assembly consists of the following four values:
+//
+//      Major Version
+//      Minor Version 
+//      Build Number
+//      Revision
+//
+// You can specify all the values or you can default the Build and Revision Numbers 
+// by using the '*' as shown below:
+// [assembly: AssemblyVersion("1.0.*")]
+[assembly: AssemblyVersion("1.0.0.0")]
+[assembly: AssemblyFileVersion("1.0.0.0")]

src/Umbraco.Tests.Benchmarks/Umbraco.Tests.Benchmarks.csproj

@@ -0,0 +1,139 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
+  <PropertyGroup>
+    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
+    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
+    <ProjectGuid>{86DEB346-089F-4106-89C8-D852B9CF2A33}</ProjectGuid>
+    <OutputType>Exe</OutputType>
+    <AppDesignerFolder>Properties</AppDesignerFolder>
+    <RootNamespace>Umbraco.Tests.Benchmarks</RootNamespace>
+    <AssemblyName>Umbraco.Tests.Benchmarks</AssemblyName>
+    <TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
+    <FileAlignment>512</FileAlignment>
+    <NuGetPackageImportStamp>
+    </NuGetPackageImportStamp>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
+    <PlatformTarget>AnyCPU</PlatformTarget>
+    <DebugSymbols>true</DebugSymbols>
+    <DebugType>full</DebugType>
+    <Optimize>false</Optimize>
+    <OutputPath>bin\Debug\</OutputPath>
+    <DefineConstants>DEBUG;TRACE</DefineConstants>
+    <ErrorReport>prompt</ErrorReport>
+    <WarningLevel>4</WarningLevel>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
+    <PlatformTarget>AnyCPU</PlatformTarget>
+    <DebugType>pdbonly</DebugType>
+    <Optimize>true</Optimize>
+    <OutputPath>bin\Release\</OutputPath>
+    <DefineConstants>TRACE</DefineConstants>
+    <ErrorReport>prompt</ErrorReport>
+    <WarningLevel>4</WarningLevel>
+  </PropertyGroup>
+  <PropertyGroup>
+    <RunPostBuildEvent>Always</RunPostBuildEvent>
+  </PropertyGroup>
+  <ItemGroup>
+    <Reference Include="BenchmarkDotNet, Version=0.9.9.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4, processorArchitecture=MSIL">
+      <HintPath>..\packages\BenchmarkDotNet.0.9.9\lib\net45\BenchmarkDotNet.dll</HintPath>
+    </Reference>
+    <Reference Include="BenchmarkDotNet.Core, Version=0.9.9.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4, processorArchitecture=MSIL">
+      <HintPath>..\packages\BenchmarkDotNet.Core.0.9.9\lib\net45\BenchmarkDotNet.Core.dll</HintPath>
+    </Reference>
+    <Reference Include="BenchmarkDotNet.Diagnostics.Windows, Version=0.9.9.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4, processorArchitecture=MSIL">
+      <HintPath>..\packages\BenchmarkDotNet.Diagnostics.Windows.0.9.9\lib\net45\BenchmarkDotNet.Diagnostics.Windows.dll</HintPath>
+    </Reference>
+    <Reference Include="BenchmarkDotNet.Toolchains.Roslyn, Version=0.9.9.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4, processorArchitecture=MSIL">
+      <HintPath>..\packages\BenchmarkDotNet.Toolchains.Roslyn.0.9.9\lib\net45\BenchmarkDotNet.Toolchains.Roslyn.dll</HintPath>
+    </Reference>
+    <Reference Include="Microsoft.CodeAnalysis, Version=1.3.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
+      <HintPath>..\packages\Microsoft.CodeAnalysis.Common.1.3.2\lib\net45\Microsoft.CodeAnalysis.dll</HintPath>
+    </Reference>
+    <Reference Include="Microsoft.CodeAnalysis.CSharp, Version=1.3.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
+      <HintPath>..\packages\Microsoft.CodeAnalysis.CSharp.1.3.2\lib\net45\Microsoft.CodeAnalysis.CSharp.dll</HintPath>
+    </Reference>
+    <Reference Include="Microsoft.Diagnostics.Tracing.TraceEvent, Version=1.0.41.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
+      <HintPath>..\packages\Microsoft.Diagnostics.Tracing.TraceEvent.1.0.41\lib\net40\Microsoft.Diagnostics.Tracing.TraceEvent.dll</HintPath>
+    </Reference>
+    <Reference Include="System" />
+    <Reference Include="System.Collections.Immutable, Version=1.1.37.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
+      <HintPath>..\packages\System.Collections.Immutable.1.1.37\lib\dotnet\System.Collections.Immutable.dll</HintPath>
+      <Private>True</Private>
+    </Reference>
+    <Reference Include="System.Core" />
+    <Reference Include="System.Data.SqlServerCe, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
+      <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.dll</HintPath>
+    </Reference>
+    <Reference Include="System.Data.SqlServerCe.Entity, Version=4.0.0.1, Culture=neutral, PublicKeyToken=89845dcd8080cc91, processorArchitecture=MSIL">
+      <HintPath>..\packages\SqlServerCE.4.0.0.1\lib\System.Data.SqlServerCe.Entity.dll</HintPath>
+    </Reference>
+    <Reference Include="System.Management" />
+    <Reference Include="System.Reflection.Metadata, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
+      <HintPath>..\packages\System.Reflection.Metadata.1.2.0\lib\portable-net45+win8\System.Reflection.Metadata.dll</HintPath>
+    </Reference>
+    <Reference Include="System.Threading.Tasks.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
+      <HintPath>..\packages\System.Threading.Tasks.Extensions.4.0.0\lib\portable-net45+win8+wp8+wpa81\System.Threading.Tasks.Extensions.dll</HintPath>
+    </Reference>
+    <Reference Include="System.Xml.Linq" />
+    <Reference Include="System.Data.DataSetExtensions" />
+    <Reference Include="Microsoft.CSharp" />
+    <Reference Include="System.Data" />
+    <Reference Include="System.Net.Http" />
+    <Reference Include="System.Xml" />
+  </ItemGroup>
+  <ItemGroup>
+    <Compile Include="LinqCastBenchmarks.cs" />
+    <Compile Include="ModelToSqlExpressionHelperBenchmarks.cs" />
+    <Compile Include="BulkInsertBenchmarks.cs" />
+    <Compile Include="Program.cs" />
+    <Compile Include="Properties\AssemblyInfo.cs" />
+    <Compile Include="XmlBenchmarks.cs" />
+  </ItemGroup>
+  <ItemGroup>
+    <None Include="App.config" />
+    <None Include="packages.config" />
+  </ItemGroup>
+  <ItemGroup>
+    <ProjectReference Include="..\Umbraco.Core\Umbraco.Core.csproj">
+      <Project>{31785bc3-256c-4613-b2f5-a1b0bdded8c1}</Project>
+      <Name>Umbraco.Core</Name>
+    </ProjectReference>
+    <ProjectReference Include="..\Umbraco.Tests\Umbraco.Tests.csproj">
+      <Project>{5d3b8245-ada6-453f-a008-50ed04bfe770}</Project>
+      <Name>Umbraco.Tests</Name>
+    </ProjectReference>
+  </ItemGroup>
+  <ItemGroup>
+    <Analyzer Include="..\packages\Microsoft.CodeAnalysis.Analyzers.1.1.0\analyzers\dotnet\cs\Microsoft.CodeAnalysis.Analyzers.dll" />
+    <Analyzer Include="..\packages\Microsoft.CodeAnalysis.Analyzers.1.1.0\analyzers\dotnet\cs\Microsoft.CodeAnalysis.CSharp.Analyzers.dll" />
+  </ItemGroup>
+  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
+  <PropertyGroup>
+    <PostBuildEvent>
+    if not exist "$(TargetDir)x86" md "$(TargetDir)x86"
+    xcopy /s /y "$(SolutionDir)packages\Microsoft.SqlServer.Compact.4.0.8854.1\NativeBinaries\x86\*.*" "$(TargetDir)x86"
+    if not exist "$(TargetDir)amd64" md "$(TargetDir)amd64"
+    xcopy /s /y "$(SolutionDir)packages\Microsoft.SqlServer.Compact.4.0.8854.1\NativeBinaries\amd64\*.*" "$(TargetDir)amd64"</PostBuildEvent>
+  </PropertyGroup>
+  <PropertyGroup>
+    <PreBuildEvent>xcopy "$(ProjectDir)"..\packages\SqlServerCE.4.0.0.1\amd64\*.* "$(TargetDir)amd64\" /Y /F /E /I /C /D
+xcopy "$(ProjectDir)"..\packages\SqlServerCE.4.0.0.1\x86\*.* "$(TargetDir)x86\" /Y /F /E /I /C /D</PreBuildEvent>
+  </PropertyGroup>
+  <Import Project="..\packages\Microsoft.Diagnostics.Tracing.TraceEvent.1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets" Condition="Exists('..\packages\Microsoft.Diagnostics.Tracing.TraceEvent.1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets')" />
+  <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
+    <PropertyGroup>
+      <ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them.  For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
+    </PropertyGroup>
+    <Error Condition="!Exists('..\packages\Microsoft.Diagnostics.Tracing.TraceEvent.1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\Microsoft.Diagnostics.Tracing.TraceEvent.1.0.41\build\Microsoft.Diagnostics.Tracing.TraceEvent.targets'))" />
+  </Target>
+  <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
+       Other similar extension points exist, see Microsoft.Common.targets.
+  <Target Name="BeforeBuild">
+  </Target>
+  <Target Name="AfterBuild">
+  </Target>
+  -->
+</Project>

src/Umbraco.Tests.Benchmarks/XmlBenchmarks.cs

@@ -0,0 +1,123 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text;
+using System.Threading.Tasks;
+using System.Xml;
+using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Configs;
+using BenchmarkDotNet.Diagnostics.Windows;
+using BenchmarkDotNet.Jobs;
+
+namespace Umbraco.Tests.Benchmarks
+{
+    [Config(typeof(Config))]
+    public class XmlBenchmarks
+    {
+        private class Config : ManualConfig
+        {
+            public Config()
+            {
+                Add(new MemoryDiagnoser());
+                //Add(ExecutionValidator.FailOnError);
+
+                //The 'quick and dirty' settings, so it runs a little quicker
+                // see benchmarkdotnet FAQ
+                Add(Job.Default
+                    .WithLaunchCount(1) // benchmark process will be launched only once
+                    .WithIterationTime(100) // 100ms per iteration
+                    .WithWarmupCount(3) // 3 warmup iteration
+                    .WithTargetCount(3)); // 3 target iteration                
+            }
+        }
+
+        [Setup]
+        public void Setup()
+        {
+            var templateId = 0;
+            var xmlText = @"<?xml version=""1.0"" encoding=""utf-8""?>
+<!DOCTYPE root[ 
+<!ELEMENT Home ANY>
+<!ATTLIST Home id ID #REQUIRED>
+<!ELEMENT CustomDocument ANY>
+<!ATTLIST CustomDocument id ID #REQUIRED>
+]>
+<root id=""-1"">
+	<Home id=""1046"" parentID=""-1"" level=""1"" writerID=""0"" creatorID=""0"" nodeType=""1044"" template=""" + templateId + @""" sortOrder=""1"" createDate=""2012-06-12T14:13:17"" updateDate=""2012-07-20T18:50:43"" nodeName=""Home"" urlName=""home"" writerName=""admin"" creatorName=""admin"" path=""-1,1046"" isDoc="""">
+		<content><![CDATA[]]></content>
+		<umbracoUrlAlias><![CDATA[this/is/my/alias, anotheralias]]></umbracoUrlAlias>
+		<umbracoNaviHide>1</umbracoNaviHide>
+		<Home id=""1173"" parentID=""1046"" level=""2"" writerID=""0"" creatorID=""0"" nodeType=""1044"" template=""" + templateId + @""" sortOrder=""2"" createDate=""2012-07-20T18:06:45"" updateDate=""2012-07-20T19:07:31"" nodeName=""Sub1"" urlName=""sub1"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1173"" isDoc="""">
+			<content><![CDATA[<div>This is some content</div>]]></content>
+			<umbracoUrlAlias><![CDATA[page2/alias, 2ndpagealias]]></umbracoUrlAlias>			
+			<Home id=""1174"" parentID=""1173"" level=""3"" writerID=""0"" creatorID=""0"" nodeType=""1044"" template=""" + templateId + @""" sortOrder=""2"" createDate=""2012-07-20T18:07:54"" updateDate=""2012-07-20T19:10:27"" nodeName=""Sub2"" urlName=""sub2"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1173,1174"" isDoc="""">
+				<content><![CDATA[]]></content>
+				<umbracoUrlAlias><![CDATA[only/one/alias]]></umbracoUrlAlias>
+				<creatorName><![CDATA[Custom data with same property name as the member name]]></creatorName>
+			</Home>
+			<Home id=""1176"" parentID=""1173"" level=""3"" writerID=""0"" creatorID=""0"" nodeType=""1044"" template=""" + templateId + @""" sortOrder=""3"" createDate=""2012-07-20T18:08:08"" updateDate=""2012-07-20T19:10:52"" nodeName=""Sub 3"" urlName=""sub-3"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1173,1176"" isDoc="""">
+				<content><![CDATA[]]></content>
+			</Home>
+			<CustomDocument id=""1177"" parentID=""1173"" level=""3"" writerID=""0"" creatorID=""0"" nodeType=""1234"" template=""" + templateId + @""" sortOrder=""4"" createDate=""2012-07-16T15:26:59"" updateDate=""2012-07-18T14:23:35"" nodeName=""custom sub 1"" urlName=""custom-sub-1"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1173,1177"" isDoc="""" />
+			<CustomDocument id=""1178"" parentID=""1173"" level=""3"" writerID=""0"" creatorID=""0"" nodeType=""1234"" template=""" + templateId + @""" sortOrder=""4"" createDate=""2012-07-16T15:26:59"" updateDate=""2012-07-16T14:23:35"" nodeName=""custom sub 2"" urlName=""custom-sub-2"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1173,1178"" isDoc="""" />
+		</Home>
+		<Home id=""1175"" parentID=""1046"" level=""2"" writerID=""0"" creatorID=""0"" nodeType=""1044"" template=""" + templateId + @""" sortOrder=""3"" createDate=""2012-07-20T18:08:01"" updateDate=""2012-07-20T18:49:32"" nodeName=""Sub 2"" urlName=""sub-2"" writerName=""admin"" creatorName=""admin"" path=""-1,1046,1175"" isDoc=""""><content><![CDATA[]]></content>
+		</Home>
+	</Home>
+	<CustomDocument id=""1172"" parentID=""-1"" level=""1"" writerID=""0"" creatorID=""0"" nodeType=""1234"" template=""" + templateId + @""" sortOrder=""2"" createDate=""2012-07-16T15:26:59"" updateDate=""2012-07-18T14:23:35"" nodeName=""Test"" urlName=""test-page"" writerName=""admin"" creatorName=""admin"" path=""-1,1172"" isDoc="""" />
+</root>";
+            _xml = new XmlDocument();
+            _xml.LoadXml(xmlText);
+        }
+
+        [Cleanup]
+        public void Cleanup()
+        {
+            _xml = null;
+        }
+
+        private XmlDocument _xml;
+
+        [Benchmark]
+        public void XmlWithXPath()
+        {
+            var xpath = "/root/* [@isDoc and @urlName='home']//* [@isDoc and @urlName='sub1']//* [@isDoc and @urlName='sub2']";
+            var elt = _xml.SelectSingleNode(xpath);
+            if (elt == null) Console.WriteLine("ERR");
+        }
+
+        [Benchmark]
+        public void XmlWithNavigation()
+        {
+            var elt = _xml.DocumentElement;
+            var id = NavigateElementRoute(elt, new[] {"home", "sub1", "sub2"});
+            if (id <= 0) Console.WriteLine("ERR");
+        }
+
+        private const bool UseLegacySchema = false;
+
+        private int NavigateElementRoute(XmlElement elt, string[] urlParts)
+        {
+            var found = true;
+            var i = 0;
+            while (found && i < urlParts.Length)
+            {
+                found = false;
+                foreach (XmlElement child in elt.ChildNodes)
+                {
+                    var noNode = UseLegacySchema
+                        ? child.Name != "node"
+                        : child.GetAttributeNode("isDoc") == null;
+                    if (noNode) continue;
+                    if (child.GetAttribute("urlName") != urlParts[i]) continue;
+
+                    found = true;
+                    elt = child;
+                    break;
+                }
+                i++;
+            }
+            return found ? int.Parse(elt.GetAttribute("id")) : -1;
+        }
+    }
+}

src/Umbraco.Tests.Benchmarks/packages.config

@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="utf-8"?>
+<packages>
+  <package id="BenchmarkDotNet" version="0.9.9" targetFramework="net45" />
+  <package id="BenchmarkDotNet.Core" version="0.9.9" targetFramework="net45" />
+  <package id="BenchmarkDotNet.Diagnostics.Windows" version="0.9.9" targetFramework="net45" />
+  <package id="BenchmarkDotNet.Toolchains.Roslyn" version="0.9.9" targetFramework="net45" />
+  <package id="Microsoft.CodeAnalysis.Analyzers" version="1.1.0" targetFramework="net45" />
+  <package id="Microsoft.CodeAnalysis.Common" version="1.3.2" targetFramework="net45" />
+  <package id="Microsoft.CodeAnalysis.CSharp" version="1.3.2" targetFramework="net45" />
+  <package id="Microsoft.Diagnostics.Tracing.TraceEvent" version="1.0.41" targetFramework="net45" />
+  <package id="Microsoft.SqlServer.Compact" version="4.0.8854.1" targetFramework="net45" />
+  <package id="SqlServerCE" version="4.0.0.1" targetFramework="net45" />
+  <package id="System.Collections" version="4.0.0" targetFramework="net45" />
+  <package id="System.Collections.Immutable" version="1.1.37" targetFramework="net45" />
+  <package id="System.Diagnostics.Debug" version="4.0.0" targetFramework="net45" />
+  <package id="System.Globalization" version="4.0.0" targetFramework="net45" />
+  <package id="System.Linq" version="4.0.0" targetFramework="net45" />
+  <package id="System.Reflection.Metadata" version="1.2.0" targetFramework="net45" />
+  <package id="System.Resources.ResourceManager" version="4.0.0" targetFramework="net45" />
+  <package id="System.Runtime" version="4.0.0" targetFramework="net45" />
+  <package id="System.Runtime.Extensions" version="4.0.0" targetFramework="net45" />
+  <package id="System.Threading" version="4.0.0" targetFramework="net45" />
+  <package id="System.Threading.Tasks" version="4.0.0" targetFramework="net45" />
+  <package id="System.Threading.Tasks.Extensions" version="4.0.0" targetFramework="net45" />
+</packages>

src/Umbraco.Tests/Models/UmbracoEntityTests.cs

@@ -1,7 +1,10 @@
 using System;
 using System.Diagnostics;
+using Moq;
 using NUnit.Framework;
+using Umbraco.Core.Logging;
 using Umbraco.Core.Models;
+using Umbraco.Core.Models.EntityBase;
 using Umbraco.Core.Serialization;
 
 namespace Umbraco.Tests.Models
@@ -9,6 +12,132 @@ namespace Umbraco.Tests.Models
     [TestFixture]
     public class UmbracoEntityTests
     {
+        [Test]
+        public void Validate_Path()
+        {
+            var entity = new UmbracoEntity();
+
+            //it's empty with no id so we need to allow it
+            Assert.IsTrue(entity.ValidatePath());
+
+            entity.Id = 1234;
+
+            //it has an id but no path, so we can't allow it
+            Assert.IsFalse(entity.ValidatePath());
+
+            entity.Path = "-1";
+
+            //invalid path
+            Assert.IsFalse(entity.ValidatePath());
+            
+            entity.Path = string.Concat("-1,", entity.Id);
+
+            //valid path
+            Assert.IsTrue(entity.ValidatePath());
+        }
+
+        [Test]
+        public void Ensure_Path_Throws_Without_Id()
+        {
+            var entity = new UmbracoEntity();
+
+            //no id assigned
+            Assert.Throws<InvalidOperationException>(() => entity.EnsureValidPath(Mock.Of<ILogger>(), umbracoEntity => new UmbracoEntity(), umbracoEntity => { }));
+        }
+
+        [Test]
+        public void Ensure_Path_Throws_Without_Parent()
+        {
+            var entity = new UmbracoEntity {Id = 1234};
+
+            //no parent found
+            Assert.Throws<NullReferenceException>(() => entity.EnsureValidPath(Mock.Of<ILogger>(), umbracoEntity => null, umbracoEntity => { }));            
+        }
+
+        [Test]
+        public void Ensure_Path_Entity_At_Root()
+        {
+            var entity = new UmbracoEntity
+            {
+                Id = 1234,
+                ParentId = -1
+            };
+
+
+            entity.EnsureValidPath(Mock.Of<ILogger>(), umbracoEntity => null, umbracoEntity => { });
+
+            //works because it's under the root
+            Assert.AreEqual("-1,1234", entity.Path);
+        }
+
+        [Test]
+        public void Ensure_Path_Entity_Valid_Parent()
+        {
+            var entity = new UmbracoEntity
+            {
+                Id = 1234,
+                ParentId = 888
+            };
+
+            entity.EnsureValidPath(Mock.Of<ILogger>(), umbracoEntity => umbracoEntity.ParentId == 888 ? new UmbracoEntity{Id = 888, Path = "-1,888"} : null, umbracoEntity => { });
+
+            //works because the parent was found
+            Assert.AreEqual("-1,888,1234", entity.Path);
+        }
+
+        [Test]
+        public void Ensure_Path_Entity_Valid_Recursive_Parent()
+        {
+            var parentA = new UmbracoEntity
+            {
+                Id = 999,
+                ParentId = -1
+            };
+
+            var parentB = new UmbracoEntity
+            {
+                Id = 888,
+                ParentId = 999
+            };
+
+            var parentC = new UmbracoEntity
+            {
+                Id = 777,
+                ParentId = 888
+            };
+
+            var entity = new UmbracoEntity
+            {
+                Id = 1234,
+                ParentId = 777
+            };
+
+            Func<IUmbracoEntity, IUmbracoEntity> getParent = umbracoEntity =>
+            {
+                switch (umbracoEntity.ParentId)
+                {
+                    case 999:
+                        return parentA;
+                    case 888:
+                        return parentB;
+                    case 777:
+                        return parentC;
+                    case 1234:
+                        return entity;
+                    default:
+                        return null;
+                }
+            };
+            
+            //this will recursively fix all paths
+            entity.EnsureValidPath(Mock.Of<ILogger>(), getParent, umbracoEntity => { });
+            
+            Assert.AreEqual("-1,999", parentA.Path);
+            Assert.AreEqual("-1,999,888", parentB.Path);
+            Assert.AreEqual("-1,999,888,777", parentC.Path);
+            Assert.AreEqual("-1,999,888,777,1234", entity.Path);            
+        }
+
         [Test]
         public void UmbracoEntity_Can_Be_Initialized_From_Dynamic()
         {

src/Umbraco.Tests/Persistence/PetaPocoCachesTest.cs

@@ -0,0 +1,198 @@
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Linq;
+using System.Threading;
+using NUnit.Framework;
+using Umbraco.Core.Models;
+using Umbraco.Core.Persistence;
+using Umbraco.Core.Services;
+using Umbraco.Tests.Services;
+using Umbraco.Tests.TestHelpers;
+using Umbraco.Tests.TestHelpers.Entities;
+
+namespace Umbraco.Tests.Persistence
+{
+    [DatabaseTestBehavior(DatabaseBehavior.NewDbFileAndSchemaPerTest)]
+    [TestFixture, NUnit.Framework.Ignore]
+    public class PetaPocoCachesTest : BaseServiceTest
+    {
+
+#if DEBUG
+        /// <summary>
+        /// This tests the peta poco caches
+        /// </summary>
+        /// <remarks>
+        /// This test WILL fail. This is because we cannot stop PetaPoco from creating more cached items for queries such as
+        ///  ContentTypeRepository.GetAll(1,2,3,4);
+        /// when combined with other GetAll queries that pass in an array of Ids, each query generated for different length
+        /// arrays will produce a unique query which then gets added to the cache.
+        /// 
+        /// This test confirms this, if you analyze the DIFFERENCE output below you can see why the cached queries grow.
+        /// </remarks>
+        [Test]
+        public void Check_Peta_Poco_Caches()
+        {
+            var result = new List<Tuple<double, int, IEnumerable<string>>>();
+
+            Database.PocoData.UseLongKeys = true;
+
+            for (int i = 0; i < 2; i++)
+            {
+                int id1, id2, id3;
+                string alias;
+                CreateStuff(out id1, out id2, out id3, out alias);
+                QueryStuff(id1, id2, id3, alias);
+
+                double totalBytes1;
+                IEnumerable<string> keys;
+                Debug.Print(Database.PocoData.PrintDebugCacheReport(out totalBytes1, out keys));
+
+                result.Add(new Tuple<double, int, IEnumerable<string>>(totalBytes1, keys.Count(), keys));
+            }
+
+            for (int index = 0; index < result.Count; index++)
+            {
+                var tuple = result[index];
+                Debug.Print("Bytes: {0}, Delegates: {1}", tuple.Item1, tuple.Item2);
+                if (index != 0)
+                {
+                    Debug.Print("----------------DIFFERENCE---------------------");
+                    var diff = tuple.Item3.Except(result[index - 1].Item3);
+                    foreach (var d in diff)
+                    {
+                        Debug.Print(d);
+                    }
+                }
+                
+            }
+
+            var allByteResults = result.Select(x => x.Item1).Distinct();
+            var totalKeys = result.Select(x => x.Item2).Distinct();
+
+            Assert.AreEqual(1, allByteResults.Count());
+            Assert.AreEqual(1, totalKeys.Count());
+        }
+
+        [Test]
+        public void Verify_Memory_Expires()
+        {
+            Database.PocoData.SlidingExpirationSeconds = 2;
+
+            var managedCache = new Database.ManagedCache();
+
+            int id1, id2, id3;
+            string alias;
+            CreateStuff(out id1, out id2, out id3, out alias);
+            QueryStuff(id1, id2, id3, alias);
+
+            var count1 = managedCache.GetCache().GetCount();
+            Debug.Print("Keys = " + count1);
+            Assert.Greater(count1, 0);
+            
+            Thread.Sleep(10000);
+
+            var count2 = managedCache.GetCache().GetCount();
+            Debug.Print("Keys = " + count2);
+            Assert.Less(count2, count1);
+        }
+
+        private void QueryStuff(int id1, int id2, int id3, string alias1)
+        {
+            var contentService = ServiceContext.ContentService;
+
+            ServiceContext.TagService.GetTagsForEntity(id1);
+
+            ServiceContext.TagService.GetAllContentTags();
+
+            ServiceContext.TagService.GetTagsForEntity(id2);
+
+            ServiceContext.TagService.GetTagsForEntity(id3);
+
+            contentService.CountDescendants(id3);
+
+            contentService.CountChildren(id3);
+
+            contentService.Count(contentTypeAlias: alias1);
+
+            contentService.Count();
+
+            contentService.GetById(Guid.NewGuid());
+
+            contentService.GetByLevel(2);
+
+            contentService.GetChildren(id1);
+
+            contentService.GetDescendants(id2);
+
+            contentService.GetVersions(id3);
+
+            contentService.GetRootContent();
+
+            contentService.GetContentForExpiration();
+
+            contentService.GetContentForRelease();
+
+            contentService.GetContentInRecycleBin();
+
+            ((ContentService)contentService).GetPublishedDescendants(new Content("Test", -1, new ContentType(-1))
+            {
+                Id = id1,
+                Path = "-1," + id1
+            });
+
+            contentService.GetByVersion(Guid.NewGuid());
+        }
+
+        private void CreateStuff(out int id1, out int id2, out int id3, out string alias)
+        {
+            var contentService = ServiceContext.ContentService;
+
+            var ctAlias = "umbTextpage" + Guid.NewGuid().ToString("N");
+            alias = ctAlias;
+
+            for (int i = 0; i < 20; i++)
+            {
+                contentService.CreateContentWithIdentity("Test", -1, "umbTextpage", 0);
+            }
+            var contentTypeService = ServiceContext.ContentTypeService;
+            var contentType = MockedContentTypes.CreateSimpleContentType(ctAlias, "test Doc Type");
+            contentTypeService.Save(contentType);
+            for (int i = 0; i < 20; i++)
+            {
+                contentService.CreateContentWithIdentity("Test", -1, ctAlias, 0);
+            }
+            var parent = contentService.CreateContentWithIdentity("Test", -1, ctAlias, 0);
+            id1 = parent.Id;
+
+            for (int i = 0; i < 20; i++)
+            {
+                contentService.CreateContentWithIdentity("Test", parent, ctAlias);
+            }
+            IContent current = parent;
+            for (int i = 0; i < 20; i++)
+            {
+                current = contentService.CreateContentWithIdentity("Test", current, ctAlias);
+            }
+            contentType = MockedContentTypes.CreateSimpleContentType("umbMandatory" + Guid.NewGuid().ToString("N"), "Mandatory Doc Type", true);
+            contentType.PropertyGroups.First().PropertyTypes.Add(
+                new PropertyType("test", DataTypeDatabaseType.Ntext, "tags")
+                {
+                    DataTypeDefinitionId = 1041
+                });
+            contentTypeService.Save(contentType);
+            var content1 = MockedContent.CreateSimpleContent(contentType, "Tagged content 1", -1);
+            content1.SetTags("tags", new[] { "hello", "world", "some", "tags" }, true);
+            contentService.Publish(content1);
+            id2 = content1.Id;
+
+            var content2 = MockedContent.CreateSimpleContent(contentType, "Tagged content 2", -1);
+            content2.SetTags("tags", new[] { "hello", "world", "some", "tags" }, true);
+            contentService.Publish(content2);
+            id3 = content2.Id;
+
+            contentService.MoveToRecycleBin(content1);
+        }
+#endif
+    }
+}

src/Umbraco.Tests/Persistence/PetaPocoExtensionsTest.cs

@@ -1,201 +1,16 @@
 using System;
 using System.Collections.Generic;
-using System.Diagnostics;
-using System.Linq;
 using System.Text.RegularExpressions;
-using System.Threading;
 using NUnit.Framework;
 using Umbraco.Core;
-using Umbraco.Core.Models;
+using Umbraco.Core.Logging;
 using Umbraco.Core.Models.Rdbms;
 using Umbraco.Core.Persistence;
-using Umbraco.Core.Services;
-using Umbraco.Tests.Services;
+using Umbraco.Core.Persistence.SqlSyntax;
 using Umbraco.Tests.TestHelpers;
-using Umbraco.Tests.TestHelpers.Entities;
 
 namespace Umbraco.Tests.Persistence
 {
-    [DatabaseTestBehavior(DatabaseBehavior.NewDbFileAndSchemaPerTest)]
-    [TestFixture, NUnit.Framework.Ignore]
-    public class PetaPocoCachesTest : BaseServiceTest
-    {
-        /// <summary>
-        /// This tests the peta poco caches
-        /// </summary>
-        /// <remarks>
-        /// This test WILL fail. This is because we cannot stop PetaPoco from creating more cached items for queries such as
-        ///  ContentTypeRepository.GetAll(1,2,3,4);
-        /// when combined with other GetAll queries that pass in an array of Ids, each query generated for different length
-        /// arrays will produce a unique query which then gets added to the cache.
-        /// 
-        /// This test confirms this, if you analyze the DIFFERENCE output below you can see why the cached queries grow.
-        /// </remarks>
-        [Test]
-        public void Check_Peta_Poco_Caches()
-        {
-            var result = new List<Tuple<double, int, IEnumerable<string>>>();
-
-            Database.PocoData.UseLongKeys = true;
-
-            for (int i = 0; i < 2; i++)
-            {
-                int id1, id2, id3;
-                string alias;
-                CreateStuff(out id1, out id2, out id3, out alias);
-                QueryStuff(id1, id2, id3, alias);
-
-                double totalBytes1;
-                IEnumerable<string> keys;
-                Debug.Print(Database.PocoData.PrintDebugCacheReport(out totalBytes1, out keys));
-
-                result.Add(new Tuple<double, int, IEnumerable<string>>(totalBytes1, keys.Count(), keys));
-            }
-
-            for (int index = 0; index < result.Count; index++)
-            {
-                var tuple = result[index];
-                Debug.Print("Bytes: {0}, Delegates: {1}", tuple.Item1, tuple.Item2);
-                if (index != 0)
-                {
-                    Debug.Print("----------------DIFFERENCE---------------------");
-                    var diff = tuple.Item3.Except(result[index - 1].Item3);
-                    foreach (var d in diff)
-                    {
-                        Debug.Print(d);
-                    }
-                }
-                
-            }
-
-            var allByteResults = result.Select(x => x.Item1).Distinct();
-            var totalKeys = result.Select(x => x.Item2).Distinct();
-
-            Assert.AreEqual(1, allByteResults.Count());
-            Assert.AreEqual(1, totalKeys.Count());
-        }
-
-        [Test]
-        public void Verify_Memory_Expires()
-        {
-            Database.PocoData.SlidingExpirationSeconds = 2;
-
-            var managedCache = new Database.ManagedCache();
-
-            int id1, id2, id3;
-            string alias;
-            CreateStuff(out id1, out id2, out id3, out alias);
-            QueryStuff(id1, id2, id3, alias);
-
-            var count1 = managedCache.GetCache().GetCount();
-            Debug.Print("Keys = " + count1);
-            Assert.Greater(count1, 0);
-            
-            Thread.Sleep(10000);
-
-            var count2 = managedCache.GetCache().GetCount();
-            Debug.Print("Keys = " + count2);
-            Assert.Less(count2, count1);
-        }
-
-        private void QueryStuff(int id1, int id2, int id3, string alias1)
-        {
-            var contentService = ServiceContext.ContentService;
-
-            ServiceContext.TagService.GetTagsForEntity(id1);
-
-            ServiceContext.TagService.GetAllContentTags();
-
-            ServiceContext.TagService.GetTagsForEntity(id2);
-
-            ServiceContext.TagService.GetTagsForEntity(id3);
-
-            contentService.CountDescendants(id3);
-
-            contentService.CountChildren(id3);
-
-            contentService.Count(contentTypeAlias: alias1);
-
-            contentService.Count();
-
-            contentService.GetById(Guid.NewGuid());
-
-            contentService.GetByLevel(2);
-
-            contentService.GetChildren(id1);
-
-            contentService.GetDescendants(id2);
-
-            contentService.GetVersions(id3);
-
-            contentService.GetRootContent();
-
-            contentService.GetContentForExpiration();
-
-            contentService.GetContentForRelease();
-
-            contentService.GetContentInRecycleBin();
-
-            ((ContentService)contentService).GetPublishedDescendants(new Content("Test", -1, new ContentType(-1))
-            {
-                Id = id1,
-                Path = "-1," + id1
-            });
-
-            contentService.GetByVersion(Guid.NewGuid());
-        }
-
-        private void CreateStuff(out int id1, out int id2, out int id3, out string alias)
-        {
-            var contentService = ServiceContext.ContentService;
-
-            var ctAlias = "umbTextpage" + Guid.NewGuid().ToString("N");
-            alias = ctAlias;
-
-            for (int i = 0; i < 20; i++)
-            {
-                contentService.CreateContentWithIdentity("Test", -1, "umbTextpage", 0);
-            }
-            var contentTypeService = ServiceContext.ContentTypeService;
-            var contentType = MockedContentTypes.CreateSimpleContentType(ctAlias, "test Doc Type");
-            contentTypeService.Save(contentType);
-            for (int i = 0; i < 20; i++)
-            {
-                contentService.CreateContentWithIdentity("Test", -1, ctAlias, 0);
-            }
-            var parent = contentService.CreateContentWithIdentity("Test", -1, ctAlias, 0);
-            id1 = parent.Id;
-
-            for (int i = 0; i < 20; i++)
-            {
-                contentService.CreateContentWithIdentity("Test", parent, ctAlias);
-            }
-            IContent current = parent;
-            for (int i = 0; i < 20; i++)
-            {
-                current = contentService.CreateContentWithIdentity("Test", current, ctAlias);
-            }
-            contentType = MockedContentTypes.CreateSimpleContentType("umbMandatory" + Guid.NewGuid().ToString("N"), "Mandatory Doc Type", true);
-            contentType.PropertyGroups.First().PropertyTypes.Add(
-                new PropertyType("test", DataTypeDatabaseType.Ntext, "tags")
-                {
-                    DataTypeDefinitionId = 1041
-                });
-            contentTypeService.Save(contentType);
-            var content1 = MockedContent.CreateSimpleContent(contentType, "Tagged content 1", -1);
-            content1.SetTags("tags", new[] { "hello", "world", "some", "tags" }, true);
-            contentService.Publish(content1);
-            id2 = content1.Id;
-
-            var content2 = MockedContent.CreateSimpleContent(contentType, "Tagged content 2", -1);
-            content2.SetTags("tags", new[] { "hello", "world", "some", "tags" }, true);
-            contentService.Publish(content2);
-            id3 = content2.Id;
-
-            contentService.MoveToRecycleBin(content1);
-        }
-    }
-
     [DatabaseTestBehavior(DatabaseBehavior.NewDbFileAndSchemaPerTest)]
     [TestFixture]
     public class PetaPocoExtensionsTest : BaseDatabaseFactoryTest
@@ -213,7 +28,7 @@ namespace Umbraco.Tests.Persistence
         }
 
         [Test]
-        public void Can_Bulk_Insert()
+        public void Can_Bulk_Insert_One_By_One()
         {
             // Arrange
             var db = DatabaseContext.Database;
@@ -234,13 +49,168 @@ namespace Umbraco.Tests.Persistence
             // Act
             using (ProfilingLogger.TraceDuration<PetaPocoExtensionsTest>("starting insert", "finished insert"))
             {
-                db.BulkInsertRecords(servers);    
+                using (var tr = db.GetTransaction())
+                {
+                    db.BulkInsertRecords(servers, tr, SqlSyntax, useNativeSqlPlatformBulkInsert:false);
+                    tr.Complete();
+                }                
             }
 
             // Assert
             Assert.That(db.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoServer"), Is.EqualTo(1000));
         }
 
+        [Test]
+        public void Can_Bulk_Insert_One_By_One_Transaction_Rollback()
+        {
+            // Arrange
+            var db = DatabaseContext.Database;
+
+            var servers = new List<ServerRegistrationDto>();
+            for (var i = 0; i < 1000; i++)
+            {
+                servers.Add(new ServerRegistrationDto
+                {
+                    ServerAddress = "address" + i,
+                    ServerIdentity = "computer" + i,
+                    DateRegistered = DateTime.Now,
+                    IsActive = true,
+                    DateAccessed = DateTime.Now
+                });
+            }
+
+            // Act
+            using (ProfilingLogger.TraceDuration<PetaPocoExtensionsTest>("starting insert", "finished insert"))
+            {
+                using (var tr = db.GetTransaction())
+                {
+                    db.BulkInsertRecords(servers, tr, SqlSyntax, useNativeSqlPlatformBulkInsert: false);
+                    //don't call complete here - the trans will be rolled back
+                }
+            }
+
+            // Assert
+            Assert.That(db.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoServer"), Is.EqualTo(0));
+        }
+
+        
+        [NUnit.Framework.Ignore("Ignored because you need to configure your own SQL Server to test thsi with")]
+        [Test]
+        public void Can_Bulk_Insert_Native_Sql_Server_Bulk_Inserts()
+        {
+            //create the db
+            var dbSqlServer = new UmbracoDatabase(
+                "server=.\\SQLExpress;database=YOURDB;user id=YOURUSER;password=YOURPASSWORD",
+                Constants.DatabaseProviders.SqlServer,
+                new DebugDiagnosticsLogger());
+
+            //drop the table
+            dbSqlServer.Execute("DROP TABLE [umbracoServer]");
+
+            //re-create it
+            dbSqlServer.Execute(@"CREATE TABLE [umbracoServer](
+	[id] [int] IDENTITY(1,1) NOT NULL,
+	[address] [nvarchar](500) NOT NULL,
+	[computerName] [nvarchar](255) NOT NULL,
+	[registeredDate] [datetime] NOT NULL CONSTRAINT [DF_umbracoServer_registeredDate]  DEFAULT (getdate()),
+	[lastNotifiedDate] [datetime] NOT NULL,
+	[isActive] [bit] NOT NULL,
+	[isMaster] [bit] NOT NULL,
+ CONSTRAINT [PK_umbracoServer] PRIMARY KEY CLUSTERED 
+(
+	[id] ASC
+)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
+)");
+            var data = new List<ServerRegistrationDto>();
+            for (var i = 0; i < 1000; i++)
+            {
+                data.Add(new ServerRegistrationDto
+                {
+                    ServerAddress = "address" + i,
+                    ServerIdentity = "computer" + i,
+                    DateRegistered = DateTime.Now,
+                    IsActive = true,
+                    DateAccessed = DateTime.Now
+                });
+            }
+
+            var sqlServerSyntax = new SqlServerSyntaxProvider();
+            using (var tr = dbSqlServer.GetTransaction())
+            {
+                dbSqlServer.BulkInsertRecords(data, tr, sqlServerSyntax, useNativeSqlPlatformBulkInsert: true);
+                tr.Complete();
+            }
+
+            // Assert
+            Assert.That(dbSqlServer.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoServer"), Is.EqualTo(1000));
+        }
+
+        [Test]
+        public void Can_Bulk_Insert_Native_Sql_Bulk_Inserts()
+        {
+            // Arrange
+            var db = DatabaseContext.Database;
+
+            var servers = new List<ServerRegistrationDto>();
+            for (var i = 0; i < 1000; i++)
+            {
+                servers.Add(new ServerRegistrationDto
+                {
+                    ServerAddress = "address" + i,
+                    ServerIdentity = "computer" + i,
+                    DateRegistered = DateTime.Now,
+                    IsActive = true,
+                    DateAccessed = DateTime.Now
+                });
+            }
+
+            // Act
+            using (ProfilingLogger.TraceDuration<PetaPocoExtensionsTest>("starting insert", "finished insert"))
+            {
+                using (var tr = db.GetTransaction())
+                {
+                    db.BulkInsertRecords(servers, tr, SqlSyntax, useNativeSqlPlatformBulkInsert: true);
+                    tr.Complete();
+                }
+            }
+
+            // Assert
+            Assert.That(db.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoServer"), Is.EqualTo(1000));
+        }
+
+        [Test]
+        public void Can_Bulk_Insert_Native_Sql_Bulk_Inserts_Transaction_Rollback()
+        {
+            // Arrange
+            var db = DatabaseContext.Database;
+
+            var servers = new List<ServerRegistrationDto>();
+            for (var i = 0; i < 1000; i++)
+            {
+                servers.Add(new ServerRegistrationDto
+                {
+                    ServerAddress = "address" + i,
+                    ServerIdentity = "computer" + i,
+                    DateRegistered = DateTime.Now,
+                    IsActive = true,
+                    DateAccessed = DateTime.Now
+                });
+            }
+
+            // Act
+            using (ProfilingLogger.TraceDuration<PetaPocoExtensionsTest>("starting insert", "finished insert"))
+            {
+                using (var tr = db.GetTransaction())
+                {
+                    db.BulkInsertRecords(servers, tr, SqlSyntax, useNativeSqlPlatformBulkInsert: true);
+                    //don't call complete here - the trans will be rolled back
+                }
+            }
+
+            // Assert
+            Assert.That(db.ExecuteScalar<int>("SELECT COUNT(*) FROM umbracoServer"), Is.EqualTo(0));
+        }
+
         [Test]
         public void Generate_Bulk_Import_Sql()
         {
@@ -263,7 +233,9 @@ namespace Umbraco.Tests.Persistence
 
             // Act
             string[] sql;
-            db.GenerateBulkInsertCommand(servers, db.Connection, out sql);
+            db.GenerateBulkInsertCommand(
+                Database.PocoData.ForType(typeof(ServerRegistrationDto)), 
+                servers, out sql);
             db.CloseSharedConnection();
 
             // Assert
@@ -295,7 +267,7 @@ namespace Umbraco.Tests.Persistence
 
             // Act
             string[] sql;
-            db.GenerateBulkInsertCommand(servers, db.Connection, out sql);
+            db.GenerateBulkInsertCommand(Database.PocoData.ForType(typeof(ServerRegistrationDto)), servers, out sql);
             db.CloseSharedConnection();
 
             // Assert

src/Umbraco.Tests/Persistence/Querying/ExpressionTests.cs

@@ -16,30 +16,54 @@ namespace Umbraco.Tests.Persistence.Querying
     [TestFixture]
     public class ExpressionTests : BaseUsingSqlCeSyntax
     {
-    //    [Test]
-    //    public void Can_Query_With_Content_Type_Alias()
-    //    {
-    //        //Arrange
-    //        Expression<Func<IMedia, bool>> predicate = content => content.ContentType.Alias == "Test";
-    //        var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IContent>();
-    //        var result = modelToSqlExpressionHelper.Visit(predicate);
+        //    [Test]
+        //    public void Can_Query_With_Content_Type_Alias()
+        //    {
+        //        //Arrange
+        //        Expression<Func<IMedia, bool>> predicate = content => content.ContentType.Alias == "Test";
+        //        var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IContent>();
+        //        var result = modelToSqlExpressionHelper.Visit(predicate);
 
-    //        Debug.Print("Model to Sql ExpressionHelper: \n" + result);
+        //        Debug.Print("Model to Sql ExpressionHelper: \n" + result);
 
-    //        Assert.AreEqual("[cmsContentType].[alias] = @0", result);
-    //        Assert.AreEqual("Test", modelToSqlExpressionHelper.GetSqlParameters()[0]);
-    //    }
+        //        Assert.AreEqual("[cmsContentType].[alias] = @0", result);
+        //        Assert.AreEqual("Test", modelToSqlExpressionHelper.GetSqlParameters()[0]);
+        //    }
+
+        [Test]
+        public void CachedExpression_Can_Verify_Path_StartsWith_Predicate_In_Same_Result()
+        {
+            //Arrange
+
+            //use a single cached expression for multiple expressions and ensure the correct output
+            // is done for both of them.
+            var cachedExpression = new CachedExpression();
+
+
+            Expression<Func<IContent, bool>> predicate1 = content => content.Path.StartsWith("-1");
+            cachedExpression.Wrap(predicate1);
+            var modelToSqlExpressionHelper1 = new ModelToSqlExpressionVisitor<IContent>();
+            var result1 = modelToSqlExpressionHelper1.Visit(cachedExpression);
+            Assert.AreEqual("upper([umbracoNode].[path]) LIKE upper(@0)", result1);
+            Assert.AreEqual("-1%", modelToSqlExpressionHelper1.GetSqlParameters()[0]);
+
+            Expression<Func<IContent, bool>> predicate2 = content => content.Path.StartsWith("-1,123,97");
+            cachedExpression.Wrap(predicate2);
+            var modelToSqlExpressionHelper2 = new ModelToSqlExpressionVisitor<IContent>();
+            var result2 = modelToSqlExpressionHelper2.Visit(cachedExpression);
+            Assert.AreEqual("upper([umbracoNode].[path]) LIKE upper(@0)", result2);
+            Assert.AreEqual("-1,123,97%", modelToSqlExpressionHelper2.GetSqlParameters()[0]);
+
+        }
 
         [Test]
         public void Can_Verify_Path_StartsWith_Predicate_In_Same_Result()
         {
             //Arrange
             Expression<Func<IContent, bool>> predicate = content => content.Path.StartsWith("-1");
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IContent>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IContent>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
-
-            Debug.Print("Model to Sql ExpressionHelper: \n" + result);
-
+            
             Assert.AreEqual("upper([umbracoNode].[path]) LIKE upper(@0)", result);
             Assert.AreEqual("-1%", modelToSqlExpressionHelper.GetSqlParameters()[0]);
         }
@@ -49,7 +73,7 @@ namespace Umbraco.Tests.Persistence.Querying
         {
             //Arrange
             Expression<Func<IContent, bool>> predicate = content => content.ParentId == -1;
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IContent>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IContent>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Model to Sql ExpressionHelper: \n" + result);
@@ -62,7 +86,7 @@ namespace Umbraco.Tests.Persistence.Querying
         public void Equals_Operator_For_Value_Gets_Escaped()
         {
             Expression<Func<IUser, bool>> predicate = user => user.Username == "hello@world.com";
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IUser>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IUser>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Model to Sql ExpressionHelper: \n" + result);
@@ -75,7 +99,7 @@ namespace Umbraco.Tests.Persistence.Querying
         public void Equals_Method_For_Value_Gets_Escaped()
         {
             Expression<Func<IUser, bool>> predicate = user => user.Username.Equals("hello@world.com");
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IUser>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IUser>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Model to Sql ExpressionHelper: \n" + result);
@@ -91,7 +115,7 @@ namespace Umbraco.Tests.Persistence.Querying
             SqlSyntaxContext.SqlSyntaxProvider = new MySqlSyntaxProvider(Mock.Of<ILogger>());
 
             Expression<Func<IUser, bool>> predicate = user => user.Username.Equals("mydomain\\myuser");
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IUser>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IUser>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Model to Sql ExpressionHelper: \n" + result);
@@ -108,7 +132,7 @@ namespace Umbraco.Tests.Persistence.Querying
             SqlSyntaxContext.SqlSyntaxProvider = new MySqlSyntaxProvider(Mock.Of<ILogger>());
 
             Expression<Func<UserDto, bool>> predicate = user => user.Login.StartsWith("mydomain\\myuser");
-            var modelToSqlExpressionHelper = new PocoToSqlExpressionHelper<UserDto>();
+            var modelToSqlExpressionHelper = new PocoToSqlExpressionVisitor<UserDto>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Poco to Sql ExpressionHelper: \n" + result);
@@ -121,7 +145,7 @@ namespace Umbraco.Tests.Persistence.Querying
         public void Sql_Replace_Mapped()
         {
             Expression<Func<IUser, bool>> predicate = user => user.Username.Replace("@world", "@test") == "hello@test.com";
-            var modelToSqlExpressionHelper = new ModelToSqlExpressionHelper<IUser>();
+            var modelToSqlExpressionHelper = new ModelToSqlExpressionVisitor<IUser>();
             var result = modelToSqlExpressionHelper.Visit(predicate);
 
             Debug.Print("Model to Sql ExpressionHelper: \n" + result);

src/Umbraco.Tests/Persistence/Repositories/ContentRepositoryTest.cs

@@ -101,7 +101,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 for (int i = 0; i < allCreated.Count; i++)
                 {
                     allCreated[i].Name = "blah" + i;
-                    //IMPORTANT testing note here: We need to changed the published state here so that 
+                    //IMPORTANT testing note here: We need to changed the published state here so that
                     // it doesn't automatically think this is simply publishing again - this forces the latest
                     // version to be Saved and not published
                     allCreated[i].ChangePublishedState(PublishedState.Saved);
@@ -109,7 +109,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 }
                 unitOfWork.Commit();
 
-                //delete all xml                 
+                //delete all xml
                 unitOfWork.Database.Execute("DELETE FROM cmsContentXml");
                 Assert.AreEqual(0, unitOfWork.Database.ExecuteScalar<int>("SELECT COUNT(*) FROM cmsContentXml"));
 
@@ -156,7 +156,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 }
                 unitOfWork.Commit();
 
-                //delete all xml                 
+                //delete all xml
                 unitOfWork.Database.Execute("DELETE FROM cmsContentXml");
                 Assert.AreEqual(0, unitOfWork.Database.ExecuteScalar<int>("SELECT COUNT(*) FROM cmsContentXml"));
 
@@ -221,7 +221,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 }
                 unitOfWork.Commit();
 
-                //delete all xml                 
+                //delete all xml
                 unitOfWork.Database.Execute("DELETE FROM cmsContentXml");
                 Assert.AreEqual(0, unitOfWork.Database.ExecuteScalar<int>("SELECT COUNT(*) FROM cmsContentXml"));
 
@@ -234,10 +234,10 @@ namespace Umbraco.Tests.Persistence.Repositories
         /// <summary>
         /// This test ensures that when property values using special database fields are saved, the actual data in the
         /// object being stored is also transformed in the same way as the data being stored in the database is.
-        /// Before you would see that ex: a decimal value being saved as 100 or "100", would be that exact value in the 
+        /// Before you would see that ex: a decimal value being saved as 100 or "100", would be that exact value in the
         /// object, but the value saved to the database was actually 100.000000.
-        /// When querying the database for the value again - the value would then differ from what is in the object. 
-        /// This caused inconsistencies between saving+publishing and simply saving and then publishing, due to the former 
+        /// When querying the database for the value again - the value would then differ from what is in the object.
+        /// This caused inconsistencies between saving+publishing and simply saving and then publishing, due to the former
         /// sending the non-transformed data directly on to publishing.
         /// </summary>
         [Test]
@@ -269,10 +269,10 @@ namespace Umbraco.Tests.Persistence.Repositories
                 var contentType = MockedContentTypes.CreateSimpleContentType("umbTextpage1", "Textpage", propertyTypeCollection);
                 contentTypeRepository.AddOrUpdate(contentType);
                 unitOfWork.Commit();
-                
+
                 // Int and decimal values are passed in as strings as they would be from the backoffice UI
                 var textpage = MockedContent.CreateSimpleContentWithSpecialDatabaseTypes(contentType, "test@umbraco.org", -1, "100", "150", dateValue);
-                
+
                 // Act
                 repository.AddOrUpdate(textpage);
                 unitOfWork.Commit();
@@ -280,7 +280,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 // Assert
                 Assert.That(contentType.HasIdentity, Is.True);
                 Assert.That(textpage.HasIdentity, Is.True);
-                
+
                 var persistedTextpage = repository.Get(textpage.Id);
                 Assert.That(persistedTextpage.Name, Is.EqualTo(textpage.Name));
                 Assert.AreEqual(100m, persistedTextpage.GetValue(decimalPropertyAlias));
@@ -342,12 +342,12 @@ namespace Umbraco.Tests.Persistence.Repositories
                 contentTypeRepository.AddOrUpdate(contentType);
                 repository.AddOrUpdate(textpage);
                 unitOfWork.Commit();
-                
+
                 // Assert
                 Assert.That(contentType.HasIdentity, Is.True);
                 Assert.That(textpage.HasIdentity, Is.True);
 
-                
+
             }
         }
 
@@ -631,9 +631,9 @@ namespace Umbraco.Tests.Persistence.Repositories
                 // Act
                 var query = Query<IContent>.Builder.Where(x => x.Name.Contains("Text"));
                 long totalRecords;
-                
+
                 try
-                {                    
+                {
                     DatabaseContext.Database.EnableSqlTrace = true;
                     DatabaseContext.Database.EnableSqlCount();
 
@@ -643,14 +643,14 @@ namespace Umbraco.Tests.Persistence.Repositories
                     Assert.AreEqual(2, result.Count());
 
                     result = repository.GetPagedResultsByQuery(query, 1, 2, out totalRecords, "title", Direction.Ascending, false);
-                    
+
                     Assert.AreEqual(1, result.Count());
                 }
                 finally
-                {                
+                {
                     DatabaseContext.Database.EnableSqlTrace = false;
                     DatabaseContext.Database.DisableSqlCount();
-                }                
+                }
             }
         }
 
@@ -761,7 +761,7 @@ namespace Umbraco.Tests.Persistence.Repositories
                 // Act
                 var query = Query<IContent>.Builder.Where(x => x.Level == 2);
                 var filterQuery = Query<IContent>.Builder.Where(x => x.Name.Contains("Page 2"));
-                
+
                 long totalRecords;
                 var result = repository.GetPagedResultsByQuery(query, 0, 1, out totalRecords, "Name", Direction.Ascending, true, filterQuery);