Uncategorized

Ghosts in the Machine?

Ghosted and unghosted pages are references not new to Microsoft Office SharePoint Server 2007, but have received increased interest as a result of their impact on upgrading from previous versions and more recently, the ability to manage pages in an unghosted state.


Introduction


Ghosted is the preferred state of pages in a site collection, ghosted pages refer to site definition files cached in memory on the server at process startup of IIS.  By caching site definition files in memory performance and scalability are improved by reducing data storage and retrieval requirements.  Ghosted pages can be reused as a result across one or many site collections on the Web application.


Unghosted pages are most commonly the result of customization through Microsoft FrontPage and/or Microsoft Office SharePoint Designer and are stored in the corresponding content database.  The contents of unghosted pages are routed through safe mode parsing in ASP.NET, which prevents server-side code from executing, and which depends entirely on the Safe Controls list specified in the web.config file of the wwwroot directory to determine which controls can be rendered at run time.  When upgrading unghosted pages in SharePoint Portal Server 2003 and or Windows SharePoint Services 2.0, some functionality is lost until the page is reset to its site definition (ghosted) and can include security trimming, navigation (Site Actions), Recycle Bin, and other core Microsoft Office SharePoint Server 2007 functionality.  The performance penalty of unghosted pages in Microsoft Office SharePoint Server 2007 is less evident when compared to previous versions due to enhancements in the .NET 2.0 runtime.


So how do pages become unghosted?


In most cases an unghosted page is the result of customization through Microsoft FrontPage and/or Microsoft Office SharePoint Designer 2007 – browser based modifications such as the manipulation of Web Parts will not cause the page to be unghosted.  In a change over Microsoft FrontPage, Microsoft Office SharePoint Designer users are presented with a prompt to indicate the page they are working with will no longer be associated with its specified site definition (see illustration).



Reghosting


In many cases users can resolve minor customizations through resetting the page to its site definition through both the Windows SharePoint Services 3.0 user interface and Microsoft Office SharePoint Designer; however, changes made within Web Part Zones will be retained after synchronizing the page with its site definition.  [Site Settings | Look and Feel | Reset to site definition]. 




In some circumstances a page cannot be reghosted, for example if the page was not based on an existing page, was created from a blank page, imported from another application or editor or (most commonly) was associated with a site definition no longer available to the server farm – this can occur if a site definition was retired or a database migration approach was implemented and the upgrade definition not made available to the target server farm.  In many cases you can simply copy a known good <page>.aspx from an alternate location and retrofit it to restore functionality.


C#


Reghosting can be also accomplished through by a server farm administration calling the RevertAllDocumentContentStreams method of the Microsoft.SharePoint namespace on each SPWeb object.


Sample Code

        static void Main(string[] args)
{
string siteUrl = “”;
SPSite Site = new SPSite(siteUrl);
foreach (SPWeb Web in Site.AllWebs)
{
Web.RevertAllDocumentContentStreams();
}
}

This code snippet is provided under the Microsoft Permissive License.


Microsoft Office SharePoint Designer 2007



  1. In Office SharePoint Designer 2007, open the Web site where the customized page resides.
  2. Right-click the page that you want to reset to the site definition, and then click Reset to Site Definition on the shortcut menu.

NOTE A copy of the customized page is placed in the same directory where the reset page resides and is named file name_copy(1), where file name is the original file name – when reghosting a page through the Windows SharePoint Services 3.0 user interface a backup copy of the page is not created.



MasterPages


If the customization was implemented on a MasterPage, the MasterPage and attached pages can be reset to their site definition through Microsoft Office SharePoint Designer 2007:


In Office SharePoint Designer 2007, open the Web site where the customized master page resides.




  1. Right-click the master page that you want to reset, such as default.master, and then click Reset to Site Definition on the shortcut menu.


    Note   By default, in the Folder List, master pages are located in the masterpages folder, which is in the _catalogs folder in the site.

    A warning message informs you that the master page’s contents will be overwritten, but that a backup copy of the current page will be created.


  2. Click Yes.

Ghosted Pages and Upgrade


Ghosted pages, though, in many cases can be upgraded safely depending on the extent of their customization (see introduction for caveats) should be considered for reghosting at the time they are upgraded.  The gradual upgrade approach allows administrators to reghost (reset to site definition) pages during the upgrade process and enables the reghosting of pages at a more granular level.  Reghosting in a gradual approach can be applied through the Upgrade user interface or optionally using the STSADM -o command line argument -Reghost in the upgrade operation, for example STSADM -o upgrade -sidebyside -url http://www.contoso.com -sitelistpath <pathtoxml> -reghost.  This granularity allows you to decide where you would like to reghost and what customizations should be retained during the upgrade.  The database migration approach and inplace approaches both permit the upgrade of unghosted pages preserving the customizations; however, these pages will be upgraded in their existing state and will need to be reghosted post-upgrade to enable Microsoft Office SharePoint Server 2007 functionality.


Locating Unghosted Pages


Now that we understand ghosted vs. unghosted pages and their implication on upgrade, how do we determine where unghosted pages exist?


PRESCAN


PRESCAN.EXE has two primary purposes:



  1. It parses and saves List definitions with the associated Lists.  SharePoint Portal Server 2003 Service Pack 2 already incorporates this feature whenever a list is modified; however, this process should be completed for all Lists, so prescan calls the SharePoint Portal Server 2003 Service Pack 2 method to persist that data.
  2. PRESCAN.EXE will report on common issues that will result in a failed upgrade; therefore, running PRESCAN.EXE, addressing reported issues, and resolving those issues, and re-running PRESCAN.EXE to verify those fixes is a best practice when planning a Microsoft Office SharePoint Server 2007/Windows SharePoint Services 3.0 upgrade.  The most commonly detected issues are:

    • Database Orphans This is a class of issue where an object exists, but the pointer with the parent object is broken and/or corrupt.   Classic examples include situations where a site exists in the content database; however, does not exist in the configuration database and a web that points to a site collection that no longer exists. 
    • Missing Site Definitions This issue is rare at best ad exists when a site collection has been removed/deleted – sites under this classification will not be upgraded and in addition those sites will not render in SharePoint Portal Server 2003/Windows SharePoint Services 2.0.

PRESCAN can be used to identify custom Site Definitions, FrontPage customizations, Web Parts, etc.

PRESCAN.EXE will report and summarize any unghosted pages occurring in your environment (see illustrations).



The example above illustrates unghosted pages discovered by PRESCAN.EXE and reported in the application log file.



The example above illustrates a summary of unghosted pages.


SQL


Alternatively SQL can be leveraged (not recommended on production databases) to generate a report of unghosted pages in your content databases (see example).



SELECT


     Docs.DirName,

     Docs.LeafName

FROM Docs WHERE

     (Docs.Type = 0)

AND

     (Docs.SetupPath IS NOT NULL) AND

     (dbo.Docs.DocFlags & 64 = 64)


Microsoft Office SharePoint Designer 2007

You can also run a report in Office SharePoint Designer 2007 to list all of the customized pages in your site.


  1. In Office SharePoint Designer 2007, open the Web site for which you want to run the report.
  2. On the Site menu, point to Reports, point to Shared Content, and then click Customized Pages.

    The report opens with all pages in the site listed, and the Customized column indicates whether content has been customized for that page.


  3. To display only pages that have been customized, click the down arrow to the right of the Customized column, and then click Yes.

    The report now displays only pages that have been customized.


Additional Resources


Joel Oleson’s To Ghost or Not to Ghost, That is the Question


MSDN – Custom Site Defintiions


Microsoft.com – SharePoint Application Templates


This Blog – PRESCAN


Microsoft.com – Upgrade Considerations for Customized Sites


Technet – Determine How to Handle Customizations

Standard
Uncategorized

Understanding PRESCAN.EXE Errors (UPDATED – May 2007)

I’ve updated my post Understanding PRESCAN.EXE Errors to include new errors and solutions. Read more here: http://blogs.technet.com/wbaer/archive/2006/12/22/prescan-errors-what-they-mean.aspx.


Excerpt:  ““An outbound zone URL is configured for something other than the default zone on virtual server http://fabrikam/, and no default zone outbound URL is defined.  This is not supported, and must be corrected before upgrading.””

Standard
Uncategorized

PRESCAN Support and Resources for Windows SharePoint Services and Office SharePoint Server

MSDN

Windows SharePoint Services


Run the pre-upgrade scan tool (Windows SharePoint Services)


http://technet2.microsoft.com/windowsserver/WSS/en/library/035a3024-bd27-4d63-9499-0f15ac00c6e61033.mspx


Issues that are reported by the pre-upgrade scan tool

http://technet2.microsoft.com/Office/en-us/library/035a3024-bd27-4d63-9499-0f15ac00c6e61033.mspx#section1


Install and run the pre-upgrade scan tool

http://technet2.microsoft.com/Office/en-us/library/035a3024-bd27-4d63-9499-0f15ac00c6e61033.mspx#section2


Error message when you try to upgrade Windows SharePoint Services 2.0 to Windows SharePoint Services 3.0: “Upgrade has encountered one or more lists that were not updated by Prescan.exe and must exit”

http://support.microsoft.com/kb/923904


Office SharePoint Server


Run the pre-upgrade scan tool (Office SharePoint Server)

http://technet2.microsoft.com/Office/en-us/library/3479e99e-0734-4237-a412-9fc42bf8bd251033.mspx?mfr=true


Issues that are reported by the pre-upgrade scan tool

http://technet2.microsoft.com/Office/en-us/library/3479e99e-0734-4237-a412-9fc42bf8bd251033.mspx#section1


Install and run the pre-upgrade scan tool

http://technet2.microsoft.com/Office/en-us/library/3479e99e-0734-4237-a412-9fc42bf8bd251033.mspx#section2


Error message when you try to upgrade Windows SharePoint Services 2.0 to Windows SharePoint Services 3.0: “Upgrade has encountered one or more lists that were not updated by Prescan.exe and must exit”

http://support.microsoft.com/kb/923904


This Blog


Understanding PRESCAN.EXE in MOSS 2007

http://blogs.technet.com/wbaer/archive/2006/05/09/understanding-prescan-exe-in-moss-2007.aspx


Understanding PRESCAN.EXE Errors

http://blogs.technet.com/wbaer/archive/2006/12/22/prescan-errors-what-they-mean.aspx

Standard
Uncategorized

Understanding PRESCAN.EXE Errors

*UPDATED 5/22/2007 

I put together this post as a follow-up to my initial PRESCAN topic “Understanding PRESCAN.EXE in MOSS 2007“; hopefully, this will provide insight into the most common PRESCAN errors administrators may encounter and remediation steps where available. These solutions may not fit every environment due to the natural differences in topologies, hosting models, versioning, and infrastructure.

Error:

There is no Web named “/<path>/<site>”.


  1. The Content Database contains one or more orphaned Webs.  A Web may not have a parent Windows SharePoint Services 2.0 site.

Find and repair orphaned Webs, Lists, and Document Libraries using STSADM -o databaserepair -url http://<server&gt; -databasename <WSS2.0_Content_Database> [-deletecorruption].  This operation requires hotfix KBs 918742 and/or 918744.  Joel Oleson has a great post on these hotfix KBs here.

Error:

Cannot locate list definition required for upgrade. Check to ensure you have all templates and language packs installed.
Error: Exception scanning web:
http://<server>/<path>/<site>
System.IO.FileNotFoundException: The system cannot find the file specified.


  1. A Windows SharePoint Services 2.0 language pack or Windows SharePoint Services 2.0 application template is not installed.


  2. Templates used by SharePoint Portal Server 2003 can be incorrectly identified during the pre-upgrade scan as custom templates unless you use the preupgradescanconfig.xml file when you perform the scan. This file contains additional logic to identify the portal site templates as standard templates used by SharePoint Portal Server 2003 rather than as custom templates based on Windows SharePoint Services 2.0.


Install language packs and application templates as required or use the preupgradescanconfig.xml file when you perform the scan.

Error:

[UpgradeDiscussionLists] [3.0.87.0] [ERROR] [12/11/2006 5:06:15 PM]: Data is Null. This method or property cannot be called on Null values.

[UpgradeDiscussionLists] [3.0.87.0] [ERROR] [12/11/2006 5:06:15 PM]: Item update failed for list 3cc63b2b-8a42-4ef3-bfa4-6e40f4827ec6, thread 20051208-1643-2400-a3fe-67a3e9495a11, ordering 20051208164324


  1. Required fields are NULL on a List within a Web and/or Windows SharePoint Services 2.0 site.  As an example, a Discussion Board may contain one or more posts with a blank Subject and/or Body field. 

Using the reported ListId, Identify the Web and/or Windows SharePoint Services 2.0 site hosting the List and correct the input as necessary.

Error:

Upgrade has encountered one or more lists that were not updated by Prescan.exe and must exit. The most likely reasons for Prescan to skip a list are covered in the Knowledge Base article at: http://go.microsoft.com/fwlink/?linkid=69958&clcid=0x409. For more information about which lists were not upgraded, see the upgrade log file at LocationOfLogFile.


  1. The content database contains one or more orphaned lists. For example, a list may not have a parent Windows SharePoint Services 2.0 site.
  2. The content database contains one or more Windows SharePoint Services 2.0 sites that use an included path. Additionally, the included path is not defined on the server.

See http://support.microsoft.com/kb/923904 for additional information.

Error:

“PRESCAN failed for site /<prefix>/<site>.  The site could not be access through the SharePoint Object Model.”


  1. A Windows SharePoint Services 2.0 Inclusion path is missing.  Confirm all inclusion paths used by Windows SharePoint Services 2.0 sites are available.

If PRESCAN fails on any Content Database you will receive an error when attempting to run the upgrade which will render as follows:

The pre-upgrade scan tool has not yet been run on this database SPContentDatabase Name=Some_Content_Database Parent=SPDatabaseServiceInstance. You must run the pre-upgrade scan tool before you can continue with the upgrade process. Run the tool from the following path: C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions12binprescan.exe.

Error: Prescan has encountered sites or lists that were not updated because they cannot be accessed using the SharePoint Products and Technologies object model. The most likely reasons for Prescan to skip a list are covered in the Knowledge Base article at: http://go.microsoft.com/fwlink/?linkid=69958&clcid=0x409.

Error:

FAILED to persist field schema of lists in web [insert URL of site]

Error: The following site has not been scanned. Id = df843563-2961-41ff-ad61-0414c67d7305 and Url = abc/TestABC

Error: The following list has not been scrubbed: Id=6bfb5f3d-fa4b-4c71-b251-0778e0e1018a, Name=Web Part Gallery, Containing Web=abc/TestABC


  1. The content database contains one or more orphaned lists. For example, a list may not have a parent Windows SharePoint Services 2.0 site.

Error:

The site http://<server>/<path>/<site> cannot be opened. The site might be corrupted. To fix this problem, delete the site and recreate it.

Error: The following site has not been scanned. Id = f85aaeee-b93e-491f-b2ff-88d449f1166f and Url = <path>/<site>
http://support.microsoft.com/kb/918744


  1. The configuration database contains one or more orphaned or inaccessible site collections. Confirm the site exists or unlock the site collection in Central Administration. If the site does not exist, recreate the site collection and run PRESCAN.

Error:


This server is not the indexer of this search application.



  1. This issue occurs if the Windows SharePoint Services 3.0 database is hosted on a separate computer that is running Microsoft SQL Server. In this situation, the Windows SharePoint Services search logon account may not have the correct permission to access or to query the Windows SharePoint Services database on the computer that is running SQL Server.  To resolve this issue visit http://support.microsoft.com/kb/923896/en-us.

Error:


An exception of type Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException was thrown.  Additional exception information: The pre-upgrade scan tool has not yet been run on all servers in the farm.  You must run the pre-upgrade scan tool before you can continue with the upgrade process.  Run the tool from the following path: c:program filescommon filesMicrosoft sharedweb server extensions12binprescan.exe.  After you have reviewed any issues found by the tool, you can run psconfig.exe again to continue the upgrade process.



  1. This issue occurs if psconfig.exe was run prior to successfully running the pre-upgrade scan tool in the server farm.  Run the pre-upgrade scan tool and resolve any reported issues before running psconfig.

To determine where PRESCAN failed you can run the attached query against your content databases, the results of the SQL statement are the Site Collections in which PRESCAN was unable to properly scan and/or access through the SharePoint Object Model.


SELECT FullUrl, Id
FROM Sites
WHERE bitflags NOT LIKE262144


Error:  “An outbound zone URL is configured for something other than the default zone on virtual server http://fabrikam/, and no default zone outbound URL is defined.  This is not supported, and must be corrected before upgrading.”



  1. This error occurs when no defaultzoneoutgoingurl is defined for the Windows SharePoint Services 2.0 virtual server; in Windows SharePoint Services 2.0 it was valid to not have a URL in the default zone while subsequently having URLs in non-default zones.  You can correct this issue by using the stsadm.exe -o addzoneurl operation or alternatively accessing the values through the Object Model (code sample below).


You can review the properties programmatically in the SPVirtualServer.Config object; a example of these values of the properties is attached below.


Example:


DefaultZoneOutgoingUrl = “”
IntranetZoneOutgoingUrl = “”
InternetZoneOutgoingUrl = “”
ExtranetZoneOutgoingUrl = “http://fabrikam/


The outgoing URLs are stored in the configuration database in the “Properties” field of “VirtualServers” table.  See an example of the “Properties” field value below:

<?xml version=”1.0″ encoding=”utf-16″?><properties><version value=”1″ /><filter><inclusions><inclusion>/</inclusion><inclusion>/sites/*</inclusion><inclusion>/personal/*</inclusion></inclusions><exclusions><exclusion>/uddi</exclusion><exclusion>/uddipublic</exclusion><exclusion>/mapaweb</exclusion></exclusions></filter><miscellaneous><property name=”securityvalidation-expire” value=”True” /><property name=”virtualserverpermsmask” value=”-1″ /><property name=”data-retrieval-services-oledb-providers” value=”DB2OLEDB;IBMDADB2;MSDAORA;OraOLEDB.Oracle;SQLOLEDB” /><property name=”extranetzoneoutgoingurl ” value=”http://someurl/” /><property name=”ssc-contact” value=”no” /><property name=”send-ad-email” value=”True” /><property name=”ssc” value=”yes” /><property name=”securityvalidation-enabled” value=”True” /><property name=”securityvalidation-timeout” value=”30″ /></miscellaneous></properties>

To access the values through the Object Model in the event te STSADM operation does not resolve the issue, use the sample code below as guidance.

SPGlobalAdmin ga = new SPGlobalAdmin();
SPVirtualServer vs = ga.OpenVirtualServer(new Uri(“http://” + System.Environment.MachineName));
vs.Config.ExtranetZoneOutgoingUrl = null;
vs.Config.Properties.Update();

You can then remove the <property name=”extranetzoneoutgoingurl ” value=”http://fabrikam/” /> property which should resolve the issue.

Standard
Administration, SharePoint

Understanding PRESCAN.EXE in MOSS 2007

Prior to upgrading to Microsoft Office SharePoint Server 2007/Windows SharePoint Services 3.0 one of the prerequisites some of you may have already noticed is running PRESCAN.EXE from the installation directory.  This post will hopefully provide some insight into PRESCAN.EXE as well as best practices on when it should be run.

PRESCAN.EXE has two primary purposes:


  1. It parses and saves List definitions with the associated Lists.  SharePoint Portal Server 2003 Service Pack 2 already incorporates this feature whenever a list is modified; however, this process should be completed for all Lists, so prescan calls the SharePoint Portal Server 2003 Service Pack 2 method to persist that data.
  2. PRESCAN.EXE will report on common issues that will result in a failed upgrade; therefore, running PRESCAN.EXE, addressing reported issues, and resolving those issues, and re-running PRESCAN.EXE to verify those fixes is a best practice when planning a Microsoft Office SharePoint Server 2007/Windows SharePoint Services 3.0 upgrade.  The most commonly detected issues are:

    • Database Orphans This is a class of issue where an object exists, but the pointer with the parent object is broken and/or corrupt.   Classic examples include situations where a site exists in the content database; however, does not exist in the configuration database and a web that points to a site collection that no longer exists. 
    • Missing Site Definitions This issue is rare at best ad exists when a site collection has been removed/deleted – sites under this classification will not be upgraded and in addition those sites will not render in SharePoint Portal Server 2003/Windows SharePoint Services 2.0.

Typically these issues will manifest as a result of failed STSADM backup and restore sequences, but also can occur at the SQL level.

Depending on the nature and growth of your environment, PRESCAN.EXE is best run one (1) week prior to the production upgrade allowing time to address issues uncovered, and again prior to the upgrade itself to ensure that those previous issues have been resolved in addition to identifying new issues, if any.

Standard