Test Regular Expressions in Real-Time with NCrunch

odns_regexToday I had to write some utility classes to parse query parameters from one search provider and transform them to work with another search provider.  Naturally this meant a lot of work with regular expressions.

Many developers dread working with regular expressions. The syntax is arcane, the patterns can be absurdly long, and debugging them is a chore. A single character change in a regex pattern, to address one edge case, can cause failures in many other inputs. Unit test covering an array of inputs is essential to have confidence in your regex code and faith that your changes after the fact don’t break existing use cases.

But how can we accelerate the initial implementation? There are an array of tools out there to help with developing regexes, but if you don’t need any of that if you have NCrunch installed to Visual Studio.

Chances are if you are a .NET developer and practice TDD, you’ve heard of NCrunch. If you’re not familiar with it, NCrunch is a tool that (among other things) runs your unit tests for you in the background, as you type. This gives you real-time feedback on your tests which is a significant accelerator, saving you the trouble of writing a test, writing some code to make the test pass, rebuilding, running the test, repeat until green. You know right away if your code is working as intended. If it isn’t, you know immediately what you broke when you see the red lights next to your test methods. Overall it’s a great tool and I highly recommend it.

Consider this example, testing an email validation pattern:

public class Email_Regex
{
	private const string EmailPattern = @".+";
	public bool IsValidEmail(string email)
	{
		return Regex.IsMatch(email, EmailPattern);
	}
}


[TestFixture]
public class Email_Regex_Tests
{
	private Email_Regex _regex;

	[SetUp]
	private void SetUp()
	{
		_regex = new Email_Regex();
	}

	[Test]
	public void IsValidEmail_SimpleEmail_IsValid()
	{
		string testEmail = "chris.sulham@gmail.com";
		Assert.IsTrue(_regex.IsValidEmail(testEmail));
	}
}

This test will pass, but clearly the pattern isn’t going to cut it in the real world. We can add another test with an invalid email, that should make the pattern not match.

[Test]
public void IsValidEmail_NonEmailInput_IsNotValid()
{
	string testEmail = "I'm not telling you my email address!";
	Assert.IsFalse(_regex.IsValidEmail(testEmail));
}

Here we can set up an array of tests against representative inputs that we’ll run through our regex. Using NCrunch, we can make edits to the pattern and see in real time whether or not our inputs are matching. As you add more tests and have to tweak the pattern, you’ll once again know immediately which test inputs are working with your changes. Seeing the lights go red the moment you add or change a character in the pattern will remove many, many painful cycles of change, build, run tests, wonder what broke everything, change again…

Developing regular expressions this way almost makes the whole process fun.  Almost.

Download NCrunch here

Sitecore User Csv Import Module

I’ve created a small module to assist with importing users into Sitecore from a csv file. The purpose of the module is to bulk-import users into Sitecore from an external FTP source, but it can also be used to push users into the system in a 1 off manner, for example if you had to move users from another system into Sitecore as part of a site migration. It also comes with an automated agent that can be configured to run regular downloads and imports of user files from an external FTP source.

Overview

The module operates off of items created in Sitecore to represent the import csv sheets. These items contain fields that let you configure how the user will be created based on the data in the sheet, as well as define a role and domain to assign the user to. The module is capable of downloading these csv sheets from an external FTP site and updating the users if the sheet is newer than the last time it was processed. The agent (disabled by default) will iterate over the items in the module’s folder to download the sheet and update the users if the sheet is newer each time it runs.  Imports can also be initiated manually using a custom ribbon button on the sheet import items from within Sitecore.

Setting Up

After downloading and installing the package to Sitecore, open /App_Config/Include/Sitecore.SharedSource.UserCsvImport.config to edit the module’s settings.  You’ll need to create the folder that will store your csv files that the module will read, this should be in the site’s /data folder. If your csv’s are hosted on an external FTP site, you can define the hostname, username and password here as well.

Using the Module

Open the Sitecore content editor, and in the master database navigate to/sitecore/system/Modules/User Csv Importer/User Csv Sheets. In this folder, you can create the User Csv Sheet items.

On the User Csv Sheet item you’ll find the following fields,

  • File Name: The make of the sheet of user data to import. If using the FTP download feature, the folder path should match the folders on the FTP server. Ex. /folder/subfolder/usersheet.csv.
  • Last Updated: The last time the sheet was processed. Clear this field to force the sheet to import again.
  • Role: The membership role to apply to this user.  If it does not exist it will be created.
  • Identity Field Name: The column in the csv to use for the user’s username.
  • Email Field Name: The column in the csv to use for the user’s email.
  • Custom Profile: The profile to use for the users being created.  The columns in the csv should map to the fields on this profile item, meaning the field names should match the names of the csv columns.  Fields that do not exist will be skipped.  See this post for how to set up custom user profiles in Sitecore.

The UserCsvImport module has been tested on Sitecore 7.2 update 3, as well as Sitecore 8.1 initial release. The module depends on a few external libraries.  The Custom Item Generator, CSVHelper for reading and parsing the CSV files, and SSH.NET for the support of secure ftp file transfers.

Download the module from the Sitecore Marketplace, or the source from GitHub.

Sitecore on Solr Cloud: Part 4 – Tuning Solr for Production

This post is part of a series of posts on setting up your Sitecore application to run with Solr Cloud. We’ll be covering the procedure for setting up a Sitecore environment using the Solr search provider, and the creation of a 3-node Solr cloud cluster. This series is broken into four parts.

For the fourth part of this series, we will discuss updating your Solr settings, using Zookeeper to push those changes to the nodes, and some tuning and optimizations we can make to Solr and Sitecore for production.  Continue reading Sitecore on Solr Cloud: Part 4 – Tuning Solr for Production

Sitecore Solr Support for Chinese Language

If you’re running Sitecore with Solr, you may have noticed crawling errors when you add versions in certain languages. A common requirement for multilingual sites is support for Chinese, which the generated Solr schema Sitecore provides does not support by default.  Fortunately, it’s relatively simple to correct this and add support for Chinese, as well as other languages that aren’t available in the default schema. Continue reading Sitecore Solr Support for Chinese Language

Dropcam Can’t Connect to Verizon FiOS Router

If you’re trying to set up Dropcam, and you have Verizon FiOS, you may be running into problems connecting it to your wi-fi.  You’re able to see the network SSID, but when you try to connect you get this error:

Wireless network not found. Please make sure the camera is within range of the router, MAC filtering is disabled, and the password is correct.

What you have to do is change your router’s security from WEP to WPA2.  If you know how to do that, great, go do it and enjoy your Dropcam.  If you don’t, here’s some steps. Continue reading Dropcam Can’t Connect to Verizon FiOS Router

Sitecore on Solr Cloud: Part 3 – Creating Your Sitecore Collection

This post is part of a series of posts on setting up your Sitecore application to run with Solr Cloud. We’ll be covering the procedure for setting up a Sitecore environment using the Solr search provider, and the creation of a 3-node Solr cloud cluster. This series is broken into four parts.

For the third part of this series, we will create our Sitecore collection, add replicas, and connect Sitecore to the collection. We’ll also go over load balancing the requests to distribute them among the Solr cloud nodes.

Continue reading Sitecore on Solr Cloud: Part 3 – Creating Your Sitecore Collection

Sitecore on Solr Cloud: Part 2 – Setting up Zookeeper and Solr

This post is part of a series of posts on setting up your Sitecore application to run with Solr Cloud. We’ll be covering the procedure for setting up a Sitecore environment using the Solr search provider, and the creation of a 3-node Solr cloud cluster. This series is broken into four parts.

For the second part of this series, we will go through the steps to set up a Zookeeper Ensemble, individual Solr nodes, and linking them together in a Solr Cloud configuration. We’ll then create Windows services to start Zookeeper and Solr automatically on each server.

Continue reading Sitecore on Solr Cloud: Part 2 – Setting up Zookeeper and Solr

Sitecore on Solr Cloud: Part 1 – Architecture

This post is part of a series of posts on setting up your Sitecore application to run with Solr Cloud. We’ll be covering the procedure for setting up a Sitecore environment using the Solr search provider, and the creation of a 3-node Solr cloud cluster. This series is broken into four parts.

For the first part of this series, we’ll go over some assumptions, prerequisites, definitions, as well as the architecture of Solr Cloud and how it interacts with a Sitecore application.

Continue reading Sitecore on Solr Cloud: Part 1 – Architecture

Preparing Your Site for Heavy Traffic

A few weeks ago one of our Sitecore clients had a piece of content hit the front page of Reddit.  Too often I see sites get an explosion of traffic, usually unexpected, and the site can’t handle the sudden spike in visitors. Fortunately, this client site’s architecture was able to handle the influx of traffic without a problem.

If your site goes down when your page views shoot up, you’ve squandered a rare opportunity. First impressions are important, and a user whose first impression of your site is instability is a user that likely won’t return. There are things you can do to prepare. My post over at Velir’s blog goes into more detail.

Seamless Sitecore Content Migration

If you’re rebuilding your site in Sitecore, chances are you are looking at a content migration at some point in the near future. CMS content migration can turn into a quagmire, but it doesn’t have to go that way.  The sooner you start planning for it, the better.

You’ll face a lot of questions and challenges during your content migration. This post aims to provide some guidance on common issues, including:

  • Questions to ask and items to define before you begin a migration
  • High-level process for conducting a content audit and deciding whether to migrate or archive content
  • Resources that you’ll need to have involved during the migration effort
  • Identifying if you’ll need a manual migration, an automated one or perhaps, a hybrid approach
  • Important tasks during a migration that are critical for success but often overlooked

While written with Sitecore in mind, and from experiences migrating content into and out of Sitecore, the principles can apply to any CMS content migration.

Check out the full post over on EContentMag.