K2 Office 365 SharePoint Online Installation and Configuration Tutorial

Introduction

Over the past few years it has become more and more common to see companies move towards K2 installations that are either completely hosted in the cloud or hosted in a combination with some servers on premise and some in the cloud. It has also been seen that many companies are taking the plunge and migrating their on premise SharePoint farms into the cloud using Microsoft’s hosted services. There are many reasons why this may be considered beneficial which unfortunately falls outside the scope of this article.

As a result of the above trends, K2 installations are now more commonly required to be installed leveraging SharePoint online as opposed to on premise. This brings about its own set of unique challenges and pitfalls, but also benefits.

This article is intended as a step-by-step guide on a typical K2 server configuration that leverages a SharePoint online / O365 instance. Although the example screenshots and details are based on an “on premise” K2 server installation, this can be easily extended / extrapolated to apply to a K2 server hosted by a cloud service such as Azure or Amazon.

Article Prerequisites

For a typical K2 environment which will be connected to SharePoint online, the following basic prerequisites will need to be met before the software can be installed and the environment configured:

  • A K2 server machine that can expose its HTTP web applications and end-points to the internet. In the below example we will be making use of a dynamic DNS service to expose a local virtual machine. Most companies will have their own domain and will be able to register a subdomain for K2 which will point to the hosted K2 server. This is at the discretion of the company involved.
  • A registered DNS record for the K2 endpoints (e.g. K2.yourcompanydomain.com).
  • A valid SSL certificate for the above domain / sub-domain. Although it may be possible to configure an environment using self-signed certificates, from experience it has been noted that this almost always leads to problems and configuration issues either during the installation or later on. It is therefore required that an official SSL certificate for the chosen domain or sub-domain be purchased from a registered certificate authority.
  • An Office 365 / SharePoint online account. This article will be making use of an Office 365 Enterprise E3 trial account. Official installations would normally make use of a registered, paid for O365 account. It may be necessary to access the company’s Azure AD (AD) management centre along with other administrative tools meaning you will need either an administrator account or access to someone who owns one. If you would like to attempt the installation using a trial O365 account, information can be found here.

Step 1 – Configure the K2 blackpearl server

The first step is to configure your K2 blackpearl server along with any required supporting technologies. In this example, because we are using a K2 virtual machine on a local network that will be exposed to the internet using a dynamic DNS account, most of the supporting technologies (Active directory, SQL, Exchange etc.) will be hosted on the K2 server itself. This allows for ease of demonstration, when configuring such an environment at a customer however, many of these technologies will be hosted on remote servers, presiding either on premise within the same local network or in the cloud. It is therefore recommended to keep a list of servers, URLS, accounts etc. in use in the environment so that these details can be used in any wizards or configurations where required.

1.1. Check the K2 server prerequisites

The first main task in any K2 installation is to first check the K2 server prerequisites. These include such things as “is there an SQL server available for the K2 databases?”, “have the required accounts been created in Active Directory to support the installation?”, “what are the mail server details for the installation?”, in addition the following are of note:

  • MSMQ configuration.
  • MSDTC configuration.
  • Account permissions.

As the full details around a typical K2 installation and configuration are outside the scope of this article, full details will not be discussed. For further reading, please reference the following articles:  Prerequisites for the K2 server role,  K2 blackpearl installation guide.

Once the prerequisites have been confirmed, the K2 installation files can be downloaded from the following location : https://portal.k2.com/downloads/bp/Default.aspx – The latest version at the time of this article is 4.6.11. It is recommended that  the combined installer be downloaded as most components will be installed in this example.

1.2. Install K2 blackpearl

Once the installation prerequisites have been confirmed, the K2 blackpearl components should be installed. In our case, the following base components are to be installed:

Blackpearl_Installation_1

It is highly recommended that the K2 workspace / identity / runtime services site as installed by the K2 installer use the same site as that which will be used by K2 SmartForms. Although it is possible to run these components on different sites with a different domain / URL, sometimes authentication problems can arise when the K2 identity site is contacted from a different domain, things seem to run more smoothly when the same domain and site is used.

Blackpearl_Install_2

Blackpearl_Install_3

It is important to note that if the K2 service account does not have an email address in Active Directory, Exchange auto discover will fail, even if the from address entered exists and has an associated AD account.

Blackpearl_Install_4

Blackpearl_Install_5

1.3. Install K2 SmartForms and K2 SmartForms control pack

There are a few different ways that the K2 SmartForms component installation can be carried out. The method presented here is one that, although it takes a bit longer, is tried and tested and seems to  always result in success in 99% of cases. The summary of these steps are as follows:

  • Install SmartForms to the same site as the K2 blackpearl services site.
  • Install SmartForms control pack.
  • Register SSL certificate (step 1.4).
  • Update bindings of K2 site in IIS – make use of registered certificate (for all k2 sites / services).
  • Re-run the K2 blackpearl configuration (from start menu) – this updates the relevant blackpearl environment fields.
  • Re-run the K2 SmartForms configuration (from start menu) – this updates the relevant SmartForms environment fields.
  • Check the SmartForms designer and runtime site web.config files to ensure that the realm values were updated successfully.

First, we will install SmartForms to the same site that we installed the K2 blackpearl web components.

SmartForms_Install_1

The same site should be used for both the designer and runtime web sites. It is also advisable that both the designer make use of the same application pool (that is separate to the application pool used by the default blackpearl web applications / services).

SmartForms_Install_2

Once the K2 SmartForms installation has completed, you can go ahead and install the SmartForms control pack. This installation is straight-forward and therefore will not be discussed.

1.4. Register and Configure the SmartForms SSL Certificates

Once SmartForms and the control pack have been installed and configured, you are now ready to install your SSL certificate on the K2 server. In most cases, this will typically be a certificate for a sub-domain of the company’s top level domain, for example bpm.mycompany.com. This certificate would usually be purchased from a trusted certificate issuer (such as Thawte) and then handed over to you for installation.

Different issuers typically provide certificates for registered domains in different formats and the installation of the certificates for use in IIS are also often different. It is recommended that you follow the guidance given from the issuer in question which is usually available on their site or through their support channels.

Once you have successfully installed the certificate on the K2 server, it should be visible in IIS under “Server Certificates”:

Self_Signed_Create

If it is not visible here you will be unable to update the bindings for the K2 site in IIS. For further information on  creating and registering a self signed certificate for testing or development purposes, please see this post.

1.5. Update blackpearl and SmartForms bindings in IIS

Once the certificate has been installed on the K2 server, it is time to reconfigure the K2 site created during the installation to use installed certificate. In this example, we will configure the K2 site to make use of a certificate installed that was issued for the domain “k2.webhop.me”.

First open IIS, click on the site named “K2” (or whatever you named the K2 site during the installation above) and then click on “Bindings” under “Actions” on the top right of the window, you should then see a dialog box similar to the following:

Bindings_Config_1

Click on the HTTPS binding and select the “Edit” button. Enter the correct host name compatible with your installed certificate, then select the installed certificate using the drop-down under “SSL certificate”. Ensure that the port is set to “443” the default for SSL. The window should look similar to the following:

Bindings_Config_2

Perform the same configuration for the HTTP binding, making sure that the port is set to 80 and the host name set to the same as what was entered for your HTTPS binding, in my case “k2.webhop.me”. The bindings summary should then look similar to the following:

Bindings_Config_3

After clicking on close, your K2 web site should now be correctly configured to run with your certificate when the site is accessed with SSL, and will also respond to requests for that host name on HTTP. It might be necessary to perform an IISRESET now for good measure. The next step is to reconfigure K2 to recognise these changes.

1.6. Reconfigure blackpearl and SmartForms to use the new bindings and URL / certificate

Because the bindings for the K2 site have now been changed, it is necessary to re-run both the blackpearl and SmartForms configuration wizards so that the K2 installation can make adjustments for these changes. First, execute the K2 blackpearl setup manager from the start menu. Once it has started, make sure to select the “Configure K2 blackpearl” option on the maintenance screen.

Continue through the configuration until you reach the “K2 workspace web site configuration” screen, click on the “Bindings” button, you should then see a window similar to the following:

Blackpearl_Config_1

As you can see above, the two new bindings which we configured are listed along with the old binding. There is a warning at the bottom explaining that the current configuration is referencing a binding that does not exist, this is to be expected. Deselect the old binding, select each of the new bindings and set the HTTPS binding as default, your screen should then look similar to the following:

Blackpearl_Config_2

After clicking on “Ok”, the following window should then be displayed:

Blackpearl_Config_3

This window can be a bit confusing. What it is basically saying is that the selected issuer will be updated with the new URL as can be seen with yellow highlighting. This is normal and it means that the setup will also update the issuer URL. Click on “Ok” and continue to the end of the configuration.

Once the blackpearl reconfiguration has completed it is necessary to then reconfigure SmartForms the same way, also by running the K2 SmartForms setup manager from the start menu. The screens for the bindings of both the designer and runtime sites should be adjusted the same way as for blackpearl with only the new bindings selected and the HTTPS binding selected as default.

With the SmartForms setup, the “Resolve Security Token Service Issuers” window looks similar to the following:

Bindings_Config_4

Select “Ok” and continue with the reconfiguration through to the end.

1.5. Confirm Access to SmartForms Designer and Runtime Sites from the Internet

Because the K2 for SharePoint app running in our SharePoint online site will be referencing the SmartForms runtime site that is running on our K2 server, it is obviously important that the URLs for K2 SmartForms be accessible from the internet. As well as being accessible, it is also important that the sites do not generate any SSL / certificate related errors as this will result in the K2 for SharePoint app registration failing later.

The K2 SmartForms designer sites should open successfully with no certificate warnings from a remote browser as follows:

Remote_Test_1

Remote_Test_2

1.6. Install the K2 for SharePoint components on the K2 server

This is a step that administrators often miss. It is required that the K2 for SharePoint setup manager (found as part of the K2 installation package) be installed on the K2 server. This allows for the components enabling K2 to interact with SharePoint 2013 to be installed.

This installation is quite straight forward. Simply execute the “Setup.exe” file inside the “K2 for SharePoint 4.6.x” folder inside the K2 installation package where blackpearl and SmartForms were located. Ensure all values are correct for your environment and run the setup manager through to completion. There should not be any issues with this installation.

Step 2 – Configure SharePoint Site and K2 for SharePoint App

The next step in the process of configuring a K2 with SharePoint online installation is to configure your SharePoint online site to recognise and make use of the K2 SmartForms end points that form part of your installation. In many ways, configuring K2 for SharePoint online is easier than its on-premise counterpart, largely because the configuration of the SharePoint app domain is done for you.

2.1. Add the K2 for SharePoint app to your SharePoint site and trust it

In order to add the K2 for SharePoint app to your SharePoint online site, follow these steps:

  • Navigate to your SharePoint online site and log in as an administrator.
  • Click on the cog icon on the top right next to the account button / user icon.
  • Select “Site contents”.
  • Click on “add an app”.
  • Click on “SharePoint Store” in the links list on the left.
  • In the search bar, search for “K2 blackpearl”.
  • An icon similar to the following should be displayed:

K2_App

  • Click on the icon, the “K2 blackpearl for SharePoint” app details page should be displayed.
  • Click on the “Add it” button.
  • On the page that follows stating “Confirm that you wish to add the app” click on the “Continue” button.
  • You should then see a page stating “You just got this app for everyone in your organization”, click on the “Return to site” button.
  • If you are prompted with another window asking “Do you trust K2 blackpearl for SharePoint” click on “Trust it”
  • Wait for the app to finish being added under site contents

2.2. Register the K2 for SharePoint app against your on-prem K2 environment

  • Under site contents, on the K2 blackpearl for SharePoint app, click on the ellipses (…) button and then select “Settings”
  •   You should then be presented with the first page of the registration wizard as follows:

K2_SharePoint_Registration_1

  • Enter your K2 SmartForms runtime URL into the “Specify a K2 URL” text box and then click on the “Next” button
  • You will then likely see another screen listing the different permissions that the K2 app will be granted, click on the “Accept” button
  • Once the URL has been validated successfully (assuming there are no problems with the security certificate or the SmartForms installation itself) the registration wizard should then run through. You may need to “Accept” certain permission requests by the application for your SharePoint online account.
  • If all was configured correctly, the registration wizard should run through to completion successfully as per the following screen:

K2_SharePoint_Config1

The following section details what has happened when the configuration wizard has run along with the changes that you may see on the K2 side.

K2 for SharePoint App Registration – Automated Configuration of K2

When the app registration is run within SharePoint, various tasks are performed for you automatically. The following actions are performed automatically by the app registration if it is able to complete successfully:

  • OAuth trust established between the K2 server (service account) and SharePoint online. The K2 server can now perform actions on the SharePoint online tenant because the service account is trusted. The K2 server should also be able to impersonate users from the O365 Azure AD tenant when performing tasks on their behalf. Different tokens are created for SharePoint as well as Azure Active Directory. All OAuth Tokens and resources are created for you automatically.
  • Azure Active Directory (AAD) security label created and configured in K2 against the AAD tenant within SharePoint. Information can be retrieved from AAD by the label using OAuth resources created in the above step.
  • K2 Trust for Azure AD claim issuer created and configured in K2 against the AAD tenant within SharePoint. This issuer is configured via trust.k2.com.
  • Claims mapping created between the AAD security label and the AAD claim issuer using settings for claims from the original issuer (https://sts.windows.net). This claim mapping allows K2 to authenticate users from the tenant AAD (the identity provider claim identifying the identity provider, e.g. sts.windows.net, and identity claim identifying the authenticated user can be decrypted and interpreted within K2).

The great thing is that if following the outlined steps correctly, K2 will be able to perform all the above configurations for you. This means you should now be able to utilize your K2 on-premise SmartForms within your O365 SharePoint instance as well as make use of all the great K2 for SharePoint features made available through the K2 for SharePoint app.

I hope this tutorial can be of some help. If you pick up on any errors or inconsistencies, please leave a comment below!

Self Signed Certificates for Testing – Using Makecert.exe in Command Prompt or PowerShell

It is often necessary to make use of a self signed certificate to test a basic development or QA environment. These types of certificates are not really production ready as certificate errors will still be thrown on a client browser. For testing though, they can be added to a user’s local certificate store manually to prevent the ugly warnings from being displayed.

Creating a certificate directly through IIS (Server node -> Server Certificates -> “Create self-signed certificate”) is not always ideal. After struggling with a few certificates that were created that way, I found that using two simple commands to create both a custom certificate authority for your certificate as well as the certificate itself yielded much better results allowing me to continue with whatever configuration or testing I had been performing.

These commands make use of the “makecert” tool to create the required entities. Detailed information on the use of the tool can be found here. Makecert is usually included as part of Visual Studio as well as with some other Microsoft software, such as SharePoint. In my case, on a machine with SharePoint installed – the tool was found at the following path:

C:\Program Files\Microsoft Office Servers\15.0\Tools\

First, to create your own custom certificate authority, navigate to the directory where makecert.exe is located and run the following command with adjustments for your requirements:

makecert.exe -n "CN=My Company Development Root CA,O=My Company,
OU=Development,L=Wallkill,S=NY,C=US" -pe -ss Root -sr LocalMachine
-sky exchange -m 120 -a sha1 -len 2048 -r

Once this has been executed, create your self signed certificate using your required domain details and the above certificate authority:

makecert.exe -n "CN=mysubdomain.mydomain.com" -pe -ss My -sr LocalMachine
-sky exchange -m 120 -in "My Company Development Root CA" -is Root
-ir LocalMachine -a sha1 -eku 1.3.6.1.5.5.7.3.1

Once this has been executed successfully, the new self-signed certificate using your custom CA will be available in IIS – visible under server certificates.

Self_Signed_Create

Quick Update and Invitation to Attend a Great K2 Webinar with K2NE

This is just a short post to update you as to why I have been so quiet recently and do a little bit of advertising for a great Webinar that my new Employer is hosting.

In May this year I joined the team at K2 Northern Europe (K2NE) as a Technical Consultant. I am currently working in the Customer Success team with a focus on remote support and Technical Advisory Services (TAS). I still perform quite a few Kerberos configurations and hope to post some more regarding that soon.

Anyway, enough about me and onto the great Webinar happening on the 11th of December 2013:

Webinar: 11th December at 10 a.m. CET (English) and at 3 p.m. CET (German).

We would like to invite you to join us for our webinar where we will showcase how to build and run business applications in SharePoint 2013 and Office 365 using K2, including:

  • Full-featured and powerful forms-based apps that use multiple data sources without writing code
  • Enable workflows across line-of-business systems
  • Tie together line-of-business systems with K2’s patented SmartObjects including native support for systems such as SAP, Microsoft CRM, SalesForce.com, Active Directory and Exchange
  • Use K2’s built-in reporting capabilities that allow for full visibility into applications

AGENDA:

  • Introduction to K2’s capabilities for SharePoint 2013 and Office 365 (15 Min.)
  • Live Demo to showcase how to build and run business applications in SharePoint 2013 (25 Min.)
  • Questions & Answers (5 Min.)

SPEAKERS’ PROFILE:

Ashley Evans, Senior Technical Specialist at K2:

Ashley has over 7 years of K2 industry experience, focusing on business development in the Nordic (and DACH) regions he is an expert in K2 and SharePoint solutions. Having worked on some of the largest and most successful European K2 projects he now works as a Senior Technical Specialist helping new and existing K2 customers and partners get the best out of their K2 and SharePoint investment.

Patrick Sender, Technical Consultant at K2:

Patrick Sender is a Technical Consultant at K2. Patrick has experience in IT solutions build on the Microsoft platform. He specialises in workflow and business process management, covering consulting, project management and implementation.

To register for this great event, please head on over to the registration form (built with K2 SmartForms) :

http://forms.k2ne.com/forms/Runtime/Runtime/Form/K2NE.EventRegistrationProcess.Forms.WebinarRegistrationPage/

Error Starting Host Server: Invalid Object Name ‘GroupProvider’ – K2 Upgrade from 0807 KB00690 to 4.5 KB001370

On upgrading K2 at a client recently (0807 KB00690 to 4.5 KB001370), I found that the host server refused to start and gave the following error:

“Error Starting Host Server: Invalid Object Name ‘GroupProvider'”. After some web searches, I found that this is a known issue that can occur when upgrading to 4.5 1370. In order to resolve this issue, the following script must first be run on the HostServer database:

Use [K2HostServer]
DELETE FROM LicensedUsers
WHERE LicensedUsers.CredentialID NOT IN (SELECT CredentialID FROM SecurityCredentialCache)

Once this script has been executed, run the HostserverAlter.sql script located in the following location:

C:\Program Files (x86)\K2 Blackpearl\Setup\Configuration\Scripts

In my case, the K2 server then started up successfully.

K2 blackpearl Legacy Database Backup Script

Although the newest releases of K2 blackpearl and blackpoint (version 4.6 and newer) consolidate all of the 14 databases that were initially used by K2 (these are  now separated using different schemas), it is still necessary in some cases to make a backup of all 14 legacy databases, very often excluding the “K2” prefix of some of the newer releases. I have created a script that creates 14 separate “.bak” files for each of the legacy databases and places them in a predefined folder. This script is an extension of Johan Liebenberg’s script located here: Backing up the K2 blackpearl Databases

DECLARE @path VARCHAR(256) -- path for backup files
DECLARE @fileName VARCHAR(256) -- filename for backup
DECLARE @fileDate VARCHAR(20) -- used for file name

-- specify database backup directory
SET @path = 'C:\Backup\'

-- specify filename format
SELECT @fileDate = CONVERT(VARCHAR(20),GETDATE(),112)

SET @fileName = @path + 'Categories' + '_' + @fileDate + '.BAK'
BACKUP DATABASE Categories
TO DISK = @fileName

SET @fileName = @path + 'Dependencies' + '_' + @fileDate + '.BAK'
BACKUP DATABASE Dependencies
TO DISK = @fileName

SET @fileName = @path + 'EnvironmentSettings' + '_' + @fileDate + '.BAK'
BACKUP DATABASE EnvironmentSettings
TO DISK = @fileName

SET @fileName = @path + 'EventBus' + '_' + @fileDate + '.BAK'
BACKUP DATABASE EventBus
TO DISK = @fileName

SET @fileName = @path + 'EventBusScheduler' + '_' + @fileDate + '.BAK'
BACKUP DATABASE EventBusScheduler
TO DISK = @fileName

SET @fileName = @path + 'HostServer' + '_' + @fileDate + '.BAK'
BACKUP DATABASE HostServer
TO DISK = @fileName

SET @fileName = @path + 'K2Server' + '_' + @fileDate + '.BAK'
BACKUP DATABASE K2Server
TO DISK = @fileName

SET @fileName = @path + 'K2ServerLog' + '_' + @fileDate + '.BAK'
BACKUP DATABASE K2ServerLog
TO DISK = @fileName

SET @fileName = @path + 'K2SQLUM' + '_' + @fileDate + '.BAK'
BACKUP DATABASE K2SQLUM
TO DISK = @fileName

SET @fileName = @path + 'SmartBox' + '_' + @fileDate + '.BAK'
BACKUP DATABASE SmartBox
TO DISK = @fileName

SET @fileName = @path + 'SmartBroker' + '_' + @fileDate + '.BAK'
BACKUP DATABASE SmartBroker
TO DISK = @fileName

SET @fileName = @path + 'SmartFunctions' + '_' + @fileDate + '.BAK'
BACKUP DATABASE SmartFunctions
TO DISK = @fileName

SET @fileName = @path + 'WebWorkflow' + '_' + @fileDate + '.BAK'
BACKUP DATABASE WebWorkflow
TO DISK = @fileName

SET @fileName = @path + 'Workspace' + '_' + @fileDate + '.BAK'
BACKUP DATABASE Workspace
TO DISK = @fileName
GO

Further information on K2 database consolidation can be found here.

K2 Issue – Symmetric Keys Missing After Database Move and Upgrade

When creating a new K2 environment, it is often easiest to backup and move the K2 database(s) across to the new environment’s SQL server and reconfiguring the database(s) to contain the new environment information rather than to redeploy all existing processes, SmartObjects and solutions into the new environment (which can often cause quite a headache with differing SmartObject GUIDs etc).

What can happen sometimes, however, is the symmetric keys used by K2 for single sign-on and other authentication purposes, go missing. Recently this happened to me when upgrading from 0807 to 4.5. This was reported in the configuration analysis after the upgrade with multiple errors similar to the following:

“Symmetric key for the <Insert Database Name here> database has not been detected.”

In order to recreate the symmetric keys for the databases where they are missing and resolve the issue, first stop the K2 server service, make a backup of all K2 database(s), and run the following scripts on each database listed in error in the configuration analysis:

DROP SYMMETRIC KEY SCSSOKey
DROP CERTIFICATE SCHostServerCert
DROP MASTER KEY

If the above commands fail, don’t fret, this just means that the symmetric keys indeed do not exist, which is why we will then run the following commands to replace them:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = ‘*CONTACT K2 SUPPORT FOR KEY*’

CREATE CERTIFICATE SCHostServerCert 
WITH SUBJECT = 'Host Server Certificate', START_DATE = '01/01/2007', EXPIRY_DATE = '01/01/2017' 
CREATE SYMMETRIC KEY SCSSOKey WITH ALGORITHM = TRIPLE_DES 
ENCRYPTION BY CERTIFICATE SCHostServerCert 

The above set of commands should succeed, resulting in the required symmetric keys being recreated for the databases.

Once you have run these commands successfully, re-analyse the nodes in the configuration analysis and you should see that they no longer display in error.

Welcome to Techmongrel

Welcome to the new Techmongrel!

Well it’s been a while since the original techmongrel was up and running, but I am back and blogging, albeit with a different focus!

During the past few years, I have worked as a K2 and SharePoint consultant in and around Johannesburg South Africa and would like to use this blog as a platform to share some of the things I have learned and continue to learn.

So if you are interested in either K2, SharePoint, InfoPath or C#, come back regularly and leave some comments!

Matthew