Interoperability for SharePoint Server 2007 and Lotus Notes or Lotus Domino

The Importance of Interoperability for SharePoint Server and Lotus Notes or Lotus Domino

The introduction of Microsoft Office SharePoint Server 2007 into organizations that have traditionally used IBM Lotus Notes and IBM Lotus Domino is increasingly common. Several different factors drive this trend. The use of Microsoft Office products is widespread in organizations; therefore, it is easy to see the advantages inherent in the close relationship between Microsoft Office and Office SharePoint Server. Different departments might introduce SharePoint Server into the organization on a trial basis, only to see its use grow dramatically. An organization might introduce SharePoint Server for a specific need, such as creating an extranet. An organization could be migrating from Lotus Notes to SharePoint Server. In all these cases, a situation exists in which SharePoint Server shares the enterprise with Lotus Notes and Lotus Domino. Whether an organization has a complete migration in progress or intends to keep SharePoint Server and Notes in the enterprise together, interoperability between the systems is important.
Typically, SharePoint solutions provide aggregation points where documents and information from several sources come together. Documents might be stored directly in SharePoint solutions in document libraries, or they might be surfaced from external repositories, sites, and file systems. Data in SharePoint solutions might come directly from lists or interoperability with external line-of-business (LOB) systems. SharePoint solutions are often implemented as intranet, extranet, or Internet sites where the business case requires access to many different sources of data.
Lotus Notes is an application platform with a portal capability. Lotus Notes comes with existing applications such as e-mail and calendaring, but also provides the capability to create custom applications. Domino is the portal capability that enables data from Notes applications to be presented in a Web environment.
Typically, Notes applications are the repositories for existing or legacy information within the organization. SharePoint Server is generally introduced into an environment where Notes already exists. This situation presents the opportunity to use existing Notes applications in new ways and to provide extended reach to users who have not traditionally used Notes. Additionally, organizations can take advantage of new capabilities in SharePoint Server that enhance existing Notes applications. Organizations that focus on interoperability ensure that users have access to important data and are provided with the capabilities that enable them to be more productive.

Support for Open Protocols

SharePoint Server and Notes or Domino support a number of open protocols that are useful as integration points. These protocols range from simple Hypertext Transfer Protocol (HTTP) requests through more complex object model calls. The integration techniques presented later in this article rely on the following underlying protocols.

HTTP

HTTP hardly needs introduction. This communication protocol that forms the foundation of the World Wide Web is fully supported in both Notes or Domino and SharePoint Server to provide access to sites, views, pages, and documents. This HTTP support enables several integration points that require little or no code. Specific HTTP implementations also enable access to e-mail and issue URL-based commands for search and retrieval.

XML

XML provides a cross-platform data format that is very useful when integrating SharePoint Server and Notes or Domino. Data from SharePoint Server lists and Notes databases can both be provided as XML documents. After this data is in XML format, you can easily repurpose it for integration between the platforms.

SOAP

SOAP is the protocol used for accessing many Web services, which ride on top of the HTTP protocol and use XML as the data format. Both SharePoint Server and Domino support the SOAP protocol for access to Web services. The available Web services include built-in (out-of-the-box) services and custom services. These services can be called to both read and write data to SharePoint Server lists and Notes databases.

COM

Notes or Domino provides a COM interface to its object model, and the associated LotusScript language can call COM objects. The Microsoft .NET Framework foundation for SharePoint Server supports calls into COM objects also. Therefore, if SharePoint Server and Notes or Domino are on the same server, direct COM calls can be made.

Common Interoperability Scenarios

When creating solutions that provide interoperability between Notes or Domino and SharePoint Server, you can use two key approaches: Domino-to-SharePoint Server/SharePoint Server-to-Domino interoperability and Notes-to-SharePoint Server/SharePoint Server-to-Notes interoperability. These approaches are supported by many different protocols and technologies that span the spectrum from no-code solutions to full-code solutions.

Domino-to-SharePoint Server/SharePoint Server-to-Domino Interoperability

Because both Domino and SharePoint Server sites are portal sites, interoperability from one to the other is possible though simple approaches such as links and IFrames. In addition, solutions using the HTTP protocol to access objects and resources work across both platforms. Essentially, this approach leaves both portals in place and shares Web-based data and content between them. Figure 1 shows a conceptual drawing of reusing Domino content in a SharePoint Server portal.
Figure 1. Using Domino content in a SharePoint Server portal

Notes-to-SharePoint Server/SharePoint Server-to-Notes Interoperability

The second approach to interoperability is to access data in a Notes database directly or to reflect SharePoint Server functionality in Notes applications. In this approach, you can use technologies such as Web services to wrap portions of data or functionality and expose it for consumption. Figure 2 shows a conceptual drawing of using a Notes database in a SharePoint Server portal.
Figure 2. Exposing Notes data in a SharePoint Server portal

Support for Standards and Technologies

Because applications use different architectures and technologies, SharePoint Server interoperability depends strongly on existing standards and technologies to expose and consume data, documents, and functionality. There are several key specifications that directly support interoperability between various platforms, and several technologies you can use to create custom interoperability solutions. Because of the support for standards, customers are able to use standards-based technology to produce or consume content with SharePoint Server. The following sections describe these specifications and technologies.

Really Simple Syndication

Really Simple Syndication (RSS) is a standard that defines a Web "feed," which contains content and metadata information for a site. Feeds are created as XML documents that provide a summary of content with metadata about the published content. These feeds are consumed by a feed reader that presents the feed information to the end user. Feed readers are available in many forms, from stand-alone readers to readers that are integrated into Internet Explorer, Microsoft Office Outlook, and SharePoint Server.

Web Services and Representational State Transfer

Web services define a standard by which clients and servers residing on separate physical computers can communicate by using HTTP. Typically, Web services are designed to use remote procedure calls (RPC) or Representational State Transfer (REST). Both types of Web services can be used for integration purposes.
Web services that are based on RPC generally use the SOAP protocol. SOAP uses underlying XML messages to communicate between the client and the server. XML messages containing the parameters for a remote procedure call are sent from client to server. The return value of the call is then sent from server to client as a new XML message.
Web services that are based on REST (often called "RESTful" services) do not use an underlying RPC layer. Instead, RESTful Web services use Uniform Resource Identifiers (URI) to provide a unique address for a given resource. The resource is then called directly over HTTP without the need to prepare or return messages. The World Wide Web itself is said to be RESTful because Web pages are returned to a browser based solely on the URI typed into the browser; no underlying message is required. RESTful Web services work in a similar way.

SharePoint Server Object Model

SharePoint Server exposes both an object model and Web services that you can use to support interoperability. The object model and Web services enable you to access nearly all content, security, and administration functionality within the Office SharePoint Server computer. The Microsoft SharePoint Products and Technologies Protocol Documentation provides technical specifications for Microsoft proprietary protocols that are implemented and used in SharePoint Products and Technologies.

Web Services for Remote Portlets

The Web Services for Remote Portlets (WSRP) specification defines a Web service interface for accessing interactive presentation-oriented Web services, and was produced through the efforts of the Web Services for Remote Portlets (WSRP) OASIS Technical Committee. WSRP is based on the gathered requirements and on the concrete proposals made to the committee. WSRP enables scenarios where a portal server (consumer) needs to aggregate content from a pre-existing remote portal (producer) with a minimum of new development work or costs.
The WSRP specification defines how to expose portlet functionality through a set of Web services. WSRP specifies how portlets should expose their functions to remote clients. When a portlet implements the Web services described in WSRP, it is said to be a WSRP producer. Figure 3 shows an architectural drawing of a user interface element in one portal being exposed to another portal through WSRP.
Figure 3. Portal interoperability with WSRP
Portals acting as remote clients discover WSRP producers either directly through a URI or through a registry. Portals that call the exposed Web services of a WSRP producer are referred to as WSRP consumers. WSRP consumers receive markup from one or more WSRP producers. This markup is then aggregated and transformed to create content for an end user in the consuming portal. SharePoint Server provides a WSRP Web Part that can act as a WSRP consumer.

Content Management Interoperability Services

Content Management Interoperability Services (CMIS) is a specification created by Microsoft, IBM, and EMC. The purpose of the specification is to simplify interoperability with enterprise content management (ECM) systems. The specification relies on the use of existing technologies and standards such as SOAP, REST, and the Atom syndication format. The specification was formally submitted to the Organization for the Advancement of Structured Information Standards (OASIS), which will manage the specification in the future. For more information, see OASIS Content Management Interoperability Services (CMIS) TC.
CMIS promises to create a standardized way in which to achieve interoperability between ECM systems. CMIS is important because repositories that implement the specification expose a standard API that simplifies publishing and consuming content. Developers can create solutions that surface content from any repository directly in a SharePoint Server portal.

Java Specification Requests

Java Specification Request (JSR) 170, entitled "Content Repository API for Java" is a specification intended to enable applications written on the Java platform to access a wide variety of content repositories through a consistent interface. The specification was written specifically to support access to repositories that have tree structures and support metadata for each node. This tree structure is known as the repository model and is independent of the underlying storage mechanism.
JSR 168 and JSR 286 are versions 1.0 and 2.0, respectively, of the Java Portlet Specification. Portlets are Web-based components that are similar to Web Parts in SharePoint Server. Portals use portlets as pluggable user-interface elements that render content on a page. JSR 168 defines the foundational portlet specification, including rules for user preference settings, data management, portlet communication through requests and responses, deployment, packaging, and security.
JSR relies solely on the Java programming language for implementation. Therefore, it is not truly a cross-platform technology; however, it can be used by some portals with which SharePoint interoperability is desired.

Content SharePoint Server Can Consume

SharePoint Server is an excellent platform to use for aggregating content from various sources. Domino or Notes exposes several technologies that SharePoint Server can use to consume content. This section addresses many of the approaches used by SharePoint Server to consume and display Notes or Domino information. Many of the examples in this section are offered as live demonstrations that you can access from the InteropTips Community site.

XML Over HTTP

Using IFrames to display content and data in SharePoint Server is a powerful and simple way to get started with Notes or Domino integration. Although it is certainly possible to use IFrames to simply reflect a page from Domino in SharePoint Server, you can also use them in more sophisticated ways to present data from Notes applications. This is made possible through Dominos support for URL commands.
URL commands enable HTTP access to Notes databases. Using URL commands, you can retrieve views of Notes data in XML, present the data in SharePoint Server, and update it in Notes. URL commands take the following form:
http://<servername>/<application.nsf>?<command>
You can use the OpenView command to return HTML from a Notes application or a Domino application. In SharePoint Server, you can use a Page Viewer Web Part to display the resulting data. Figure 4 shows an example of Notes data displayed in SharePoint Server. This is a simple Page Viewer Web Part set to the address http://www.msdomino.net/orders.nsf/customer%20orders?OpenView.
Figure 4. Using the OpenView command with a Page Viewer Web Part
You can use the OpenDocument command to open a Notes document and display it in SharePoint Server. A common use of the OpenDocument command is to display the Notes e-mail application in SharePoint Server. Figure 5 shows an example of a Notes inbox in SharePoint Server. The sample uses a PageViewer Web Part set to the address http://www.msdomino.net/mail/juser.nsf/iNotes/Welcome/?OpenDocument.
Figure 5. Using the OpenDocument command to display e-mail in SharePoint Server
Using the ReadViewEntries command, you can return raw XML data instead of formatted HTML. This is useful when your application wants direct access to the data in a Notes application. Figure 6 shows an example of a Page Viewer Web Part rendering XML from a Notes application in SharePoint Server. The sample uses a Page Viewer Web Part set to the address http://www.msdomino.net/orders.nsf/customer%20orders?ReadViewEntries.
Figure 6. Using the ReadViewEntries command to retrieve XML data
The OpenAgent command enables you to invoke a LotusScript agent by using a URL command. Agents can perform many tasks on a Notes or Domino platform, including retrieving, updating, creating, and deleting data. You can use agents in conjunction with custom Web Parts to retrieve and manipulate Notes data.
Using custom Web Parts enables you to control the format and usage of the data in ways that are not possible in an IFrame alone. Figure 7 shows an example of using a Data View Web Part to format XML data returned from a custom aganet command. The data View Web Part accepts raw XML and then formats it by using an XSLT transformation. The Data View Web Part was added to the page by using Microsoft Office SharePoint Designer 2007 and set to receive data from the address http://www.msdomino.net/orders.nsf/ORDERSXML?OpenAgent. ORDERSXML is a LotusScript agent that creates an XML file suitable for use with the Data View Web Part.
Figure 7. Formatting raw XML data with the Data View Web Part

Excel and Excel Services

Microsoft Office Excel 2007 is widely used as a data analysis and reporting tool. By combining the XML over HTTP functionality presented in the previous section, you can create solutions that use Notes data in Office Excel worksheets, charts, and PivotTable reports. This process can be as simple as using the Data tab in Excel to connect to a LotusScript agent. Figure 8 shows the New Web Query dialog box referencing the address http://www.msdomino.net/orders.nsf/SHORTORDERSXML?OpenAgent.
Figure 8. Importing XML data into Excel 2007
After you import data into Excel 2007, you can analyze, format, and graph the data. Figure 9 shows the imported data filtered for the United States, Canada, and Mexico. Additionally, data bars are used to indicate relative values of freight.
Figure 9. Creating a report in Excel 2007
After creating a report in Excel 2007, you can upload the report into a SharePoint Server document library where it can be displayed by using Excel Services. Excel Services enables worksheets to be displayed as a Web page in the browser. Although this capability is interesting, it becomes compelling when used as the basis for a dashboard in the SharePoint Server Report Center.
A dashboard in the SharePoint Server Report Center is a combination of an Excel Services rendering of the worksheet data coupled with a key performance indicator (KPI) list. You can publish a worksheet to Excel Services directly through Excel 2007. To do so, select Publish on the main menu. Before you can publish the worksheet to Excel Services, however, you must remove the XML map that was created when the data was imported. XML maps are not supported under Excel Services, so the worksheet will not render until the XML map is removed. You remove an XML map through the Developer tab in Excel 2007.
After you publish the worksheet, you can use it as the basis for a dashboard in Excel Services. Creating a dashboard in the Report Center gives you the opportunity to select a worksheet to use and to add a KPI list. Figure 10 shows a complete dashboard in the Report Center using Notes data.
Figure 10. Creating a dashboard in Excel Services

Access and XSLT

Microsoft Office Access 2007 is a desktop database that you can use to import and transform data. In addition, Office Access 2007 offers tight integration with SharePoint Server, enabling it to be used to create and consume SharePoint list data. Using a combination of Access 2007, SharePoint Server 2007, and XSLT, you can import Notes data, transform it, and create SharePoint lists.
The process is accomplished in Access 2007 by importing data from a Notes database. During the import process, you can apply an XSLT transformation to the data to create the structure you want for the SharePoint list. Figure 11 shows XML data imported from http://www.msdomino.net/orders.nsf/customer%20orders?readviewentries and transformed by using the style sheet located at http://www.msdomino.net/firstpage.nsf/b29a5df9173b3feb88256e78000a0589/8c85c3b4013dfd448625754b000175ad/$FILE/ReadViewEntries-XSLT-XML.xsl. After the data is transformed into an Access 2007 table, it can be exported directly into a SharePoint list.
Figure 11. Notes data after import and transformation

Web Services

By using URL commands, you can also expose Notes or Domino data through Web services. Domino 7 introduced a new Web services design element, similar to an agent that enables the creation of a Web service. Figure 12 shows the WSDL for a sample Web service located at http://www.msdomino.net/Orders.nsf/order_update?wsdl.
Figure 12. Sample Web service WSDL
After a Web service is created, it can be consumed by any .NET Framework client, including Windows Forms and ASPX pages. A common way to consume Web services and use the data in SharePoint Server is to write a custom Web Part. Figure 13 shows a custom Web Part in SharePoint Server for managing a single record in a Notes database that is based on the WSDL presented in Figure 11.
Figure 13. Custom Web Part consuming a Web service

InfoPath and InfoPath Forms Services

Microsoft Office InfoPath 2007 forms are commonly used in SharePoint Server for data entry and workflow applications. Solutions based on Microsoft Office InfoPath forms can be deployed to Form Libraries for use with the Office InfoPath client software, or can be targeted for use with InfoPath Forms Services. InfoPath Forms Services enables a user to fill out InfoPath forms by using a browser, even if the Office InfoPath client is not present. Both InfoPath and InfoPath Forms Services solutions can take advantage of Lotus Notes data by using the Web services approach described in the previous section.
You can use InfoPath to create a form based on a Web service. By using this functionality, you can create a form by referencing a Web service created in Domino. The form is created using a simple wizard and results in a query interface that allows access to the operations supported by the Web service. After these forms are created, you can publish them to a Form Library in SharePoint Server with or without the use of InfoPath Forms Services. Figure 14 shows an InfoPath form that is created based on the Web service found at http://www.msdomino.net/products.nsf/manage_products?wsdl.

Note:
The Domino Web service used in InfoPath must have its SOAP message format set to Wrapped.

Figure 14. InfoPath form using a Domino Web service

Business Data Catalog

The Business Data Catalog is a feature found in the Enterprise Edition of Microsoft Office SharePoint Server 2007. The purpose of the Business Data Catalog is to enable SharePoint Server to use data from external LOB systems, such as Notes databases exposed through Web services. To connect to an external system, the Business Data Catalog requires an application definition file that describes the target system and how to connect with it. After this application definition file is created and loaded into the Business Data Catalog, data can be consumed from the target system.
After establishing a data connection, SharePoint Server can use data from the Notes database in several ways:

• A set of Business Data Catalog Web Parts is included with SharePoint Server that enables users to perform wildcard lookups against the data source, and then display detailed information about any single item.
  
• The Business Data Catalog data can be used to create a pick list for a column in a SharePoint list. A list of products that was sourced from a Notes database, for example, could be presented.
  
• The Business Data Catalog data can be used to supplement profile information in SharePoint Server.
  
• Business Data Catalog data can be searched by using the Enterprise Search capability of SharePoint Server.
  

Figure 15 shows data from the Web service found at http://www.msdomino.net/products.nsf/manage_products?wsdl used in a lookup list column in SharePoint Server.
Figure 15. Accessing a Domino Web service through the Business Data Catalog (BDC)

Content SharePoint Server Can Produce

In the same way that Microsoft Office applications can consume content from Lotus Notes, Office applications can also produce content for consumption in Domino. Just as SharePoint Server supports consuming data through several technology standards, it also supports exposing data through these standards. This section addresses ways to make SharePoint content available to Domino portals.

Links

SharePoint portals consist of pages that can be accessed through hyperlinks. Therefore, other Domino portals can use simple lists of hyperlinks to access various pages within the SharePoint portal. Some of these pages might also support QueryString arguments that can be used with the links. For example, a portal could link to the SharePoint search page and provide a keyword within the QueryString argument so that the link automatically executes a search.

IFrames

SharePoint page content can also be rendered within an IFrame. Domino portals can use a variety of technologies, such as portlets, to render IFrames that reference SharePoint pages. End users still need permission to view the content, however, even when the content is rendered in an IFrame.

RSS

All lists in SharePoint Server and search results support RSS. Users select to subscribe to lists by using the Action menu on the list toolbar. Users can also subscribe to search results by using an icon that appears on the search page. All SharePoint RSS feeds are authenticated, which means that users must have a feed reader that supports authentication to view SharePoint content.

Web Services for Remote Portlets

Although SharePoint Server does not provide a built-in WSRP producer, Microsoft has developed the WSRP Toolkit for SharePoint Server 2007. The toolkit includes a white paper that describes how to enable interoperability between SharePoint Server and portals based on technologies that support WSRP. This document details two methods for adding WSRP producer functionality to SharePoint Server, and briefly explores other interoperability options through data integration by using industry standards.

Web Services

SharePoint Server exposes several Web services that Domino portals can consume. These Web services provide methods for use with sites, lists, users, and search. Using these Web services enables Notes databases to consume SharePoint content and push data into SharePoint lists and libraries.
A common scenario under which SharePoint Web services are used involves uploading of data from a Notes form into a SharePoint library. Data from a Notes form can be converted to a Microsoft Office Word 2007 document by using Office Open XML file formats. You can then upload the resulting Office Word 2007 document into a SharePoint document library by using an exposed SharePoint Web service. This enables end users to continue initiating business processes through Notes forms while using SharePoint capabilities such as workflow.
Although the existing Web services do provide a significant amount of functionality, you can also write custom Web services that wrap the SharePoint Server object model. Wrapping the SharePoint Server object model opens up a significant amount of functionality that is not available through the native SharePoint Web services. Figure 16 shows a conceptual drawing of Notes integration with SharePoint Web services.
Figure 16. Integration with Web services

RPC Services

Remote Procedure Call (RPC) is a protocol layered on top of HTTP that enables a client application to communicate with a server running SharePoint Server. RPC calls made from the client take the form of an HTTP POST request to send methods to SharePoint Server. These POST requests send XML messages formatted by using Collaborative Application Markup Language (CAML). CAML is an XML language that is specific to Microsoft SharePoint Products and Technologies, and forms the underlying message for RPC method calls to request documents, update lists, and perform other operations.

Security Considerations

When integrating Lotus Notes and SharePoint Server, it is important to recognize that the security models used by each product are different. Notes uses a proprietary security model where authentication and authorization is accomplished through a separate provider. SharePoint Server can use Windows authentication (either NTLM or Kerberos protocol) or a forms authentication provider. As a result, you must consider how to allow authentication when accessing data in each system.
In a simple scenario, you can use a manual log on (such as when accessing data through Word, Excel, or Access). In more complex scenarios, you can provide a programmatic solution such as one using impersonation. Finally, consider the use of the SharePoint Single Sign-On service, which can associate non-Windows–based credentials for Notes with Windows credentials used in SharePoint Server. Additionally, it is important to understand that SharePoint Server provides security at the site level, list level, and item level. This means that you must be aware of how the security model is structured within a SharePoint site before creating integration solutions.

Conclusion

This article examines interoperability between Microsoft Office SharePoint Server 2007 and Lotus Notes or Lotus Domino. Many organizations that have traditionally used Notes and Domino have also introduced the Microsoft Office SharePoint Server 2007 platform into their organizations. The existing investment in Lotus Notes often means that the platforms will exist side by side for a period of time. Although both platforms are in use, organizations can take advantage of the interoperability technologies and techniques presented in this article to help ensure that both platforms provide maximum value.