How To Manually Deploy Sitecore Xdb Collection Databases

Every once in a while, there comes a time when your automation fails, or it doesn’t do exactly what you need it to do, and you’re faced with a choice. Do you alter the automation scripts or do you just do it by hand? Recently I faced a situation when installing Sitecore 9.1 where SIF just didn’t create the collection databases. Rather than spend time debugging SIF, I decided to just create the collection databases by hand. Turns out, it wasn’t that hard.

My first instinct was to pull the .dacpac files from the Sitecore provided WDPs (web deploy packages) and deploy them to SQL myself. Job’s done right? But how do we create that ShardMapManagerDb database? There’s no package for that.

If you watch what SIF does during an installation, you may have noticed it runs something called the SqlShardingDeploymentTool.exe. Turns out, that tool does most of the work for us, and we just have to invoke it with the right parameters. Credit to this excellent post from Kelly Rusk that explains what these parameters do. Here are the steps to do it yourself.

  1. Find the \App_Data\collectiondeployment folder in your XConnect instance. You can also extract this from the XConnect WDP package.
  2. Create a collection_user in SQL. In my case SIF did this for me (despite not creating the collection DBs), but you will need to make one if doing this completely from scratch.
  3. Prepare your command in notepad. You’ll need to pass the tool a lot of parameters, and feel free to reference the post linked earlier to understand them. Here’s one I used for reference:
    Sitecore.Xdb.Collection.Database.SqlShardingDeploymentTool.exe /operation "create" /connectionstring "Data Source=.\SQLEXPRESS;User Id=collection_user;Password=Asdf1234;Integrated Security=false;Timeout=30" /dbedition "Standard" /shardMapManagerDatabaseName "SC911.Xdb.Collection.Database.Sql.ShardMapManagerDb" /shardMapNames "ContactIdShardMap,DeviceProfileIdShardMap,ContactIdentifiersIndexShardMap" /shardnumber "2" /shardnameprefix "SC911.Xdb.Collection.Database.Sql.Shard" /shardnamesuffix "" /dacpac "Sitecore.Xdb.Collection.Database.Sql.dacpac" /log "tool.log"
    
  4. Run a command prompt as administrator and navigate to the directory with the tool. Execute your prepared command. If all goes well you’ll see a tool.log in that folder that ends with *** Everything is done. Sitecore xDB Collection SQL Sharding Deployment Tool is about to end its work. *** Make sure you see your databases in SQL.
  5. In the tool directory, you’ll see a few .sql scripts. We need to execute 2 of them to grant the proper permissions to the collection_user we created. In all cases, substitue the $(Username) variables with the name of your collection user.
  6. CreateShardApplicationDatabaseUser.sql should be run against each of the shard databases. CreateShardManagerApplicationDatabaseUser.sql should be run against the shard map manager database. The other SQL scripts in this directory are unnecessary, they execute a subset of the commands in these two scripts.

If all went well you should be able to see XConnect connecting to and logging data to the newly created collection DBs. Happy Sitecoreing!

Modify Sitecore Install Framework Packages for Azure SQL

Unfrozen Caveman LawyerSitecore 9 is here, it’s in our lives, and we’re at the point where the projects we started at the beginning of the year are getting ready to roll out. That means we need to get our production environments ready. If you’re coming from the Sitecore 8.x and earlier world, this can be a challenge. There’s new databases, the xConnect service, security and certificate requirements, and of course our friend Solr is mandatory now. We have a new tool to help us get through all this, the Sitecore Install Framework (or SIF). It’s supposed to help us by automating our install steps, if you know how to use it.

Fortunately, Sitecore has really stepped up their documentation, especially with version 9. There’s a detailed guide on installing Sitecore 9, which covers a single instance (probably a local developer environment) and a scaled out production instance. However, when they say scaled out , they mean scaled out. There’s a script for every possible server role. In the real world, our environments don’t match what’s exactly in the documentation. For example, we often combine roles, or share hardware. We need to make some adjustments, and that’s when we start to go off the map.

Continue reading Modify Sitecore Install Framework Packages for Azure SQL