Category: XM Cloud

Real-time Search Updates with Experience Edge Webhooks: Part 1

In a previous post, we went over how to use GraphQL and a custom Next.js web service to crawl and index our Sitecore XM Cloud content into a search provider. That crawler runs on a schedule, so what happens when your authors update their content? They’ll need to wait for the next run of the crawler to see their content in the search index. This is a step back in capabilities from legacy Sitecore XP, which updated indexes at the end of every publish.

It’s possible to recreate this functionality using Experience Edge webhooks. Experience Edge offers quite a few webhook options (see the list here). To enable near real-time updates of our search index, we’ll use the ContentUpdated webhook, which fires after a publish to Edge from XM Cloud finishes. Let’s take a look at an example payload from that webhook:

{
  "invocation_id": "56a95d51-b2af-496d-9bf1-a4f7eea5a7cf",
  "updates": [
    {
      "identifier": "FA27B51EE4394CBB89F8F451B13FF9DC",
      "entity_definition": "Item",
      "operation": "Update",
      "entity_culture": "en"
    },
    {
      "identifier": "80471F77728345D4859E0FD004F42FEB",
      "entity_definition": "Item",
      "operation": "Update",
      "entity_culture": "en"
    },
    {
      "identifier": "80471F77728345D4859E0FD004F42FEB-layout",
      "entity_definition": "LayoutData",
      "operation": "Update",
      "entity_culture": "en"
    },
    {
      "identifier": "7C77981F5CE04246A98BF4A95279CBFB",
      "entity_definition": "Item",
      "operation": "Update",
      "entity_culture": "en"
    },
    {
      "identifier": "FFF8F4010B2646AF8804BA39EBEE8E83-layout",
      "entity_definition": "LayoutData",
      "operation": "Update",
      "entity_culture": "en"
    }
  ],
  "continues": false
}

As you can see, we have item data here and layout data. The layout data is what we’re interested in, as this represents our actual web pages, and that is what we want to index.

The general process is as follows:

  1. Set up a receiver for this webhook. We’ll do this with a Next.js function.
  2. Loop over the webhook payload and for each piece of LayoutData, then make a GraphQL query to get the field data from Experience Edge.
  3. Finally, roll up the field data into a JSON object and push it to our search index.

Let’s start by setting up our webhook. You’ll need to create an Edge administration credential in the XM Cloud Deploy app. Make note of the Client ID and Client Secret. The secret will only be displayed once, so if you lose it you will need to create new credentials.

The next step is to create an auth token, you’ll need this to perform any Experience Edge administration actions. I used the ThunderClient plugin for Visual Studio Code to interact with the Sitecore APIs. To create an auth token, make a post request to https://auth.sitecorecloud.io/oauth/token with the following form data, using the client id and secret you just created in XM Cloud:

You’ll get back a json object containing an access token. This token is needed to be sent along with any API requests to Experience Edge. This token is passed as a Bearer Token in the Auth header. We can test it with a simple GET request that will list all the webhooks in this Edge tenant.

You should get back a json object containing a list of all the webhooks currently set up in your tenant (which is likely none to begin). The auth tokens expire after a day or so. If you get a message like edge.cdn.4:JWT Verification failed in your response, you have a problem with your token and should generate a new one.

Next let’s create our ContentUpdated webhook. You’ll need something to receive the webhook. Since we haven’t created our function in Next.js yet, we can use a testing service like Webhook.site. Create a POST request to https://edge.sitecorecloud.io/api/admin/v1/webhooks with the following body:

The important parameters here are uri and executionMode. The uri is where the webhook will be sent, in this case our testing endpoint at webhook.site. The execution mode OnUpdate indicates this will fire when content is Updated. (Note: There are separate webhooks for create and delete, which you will probably need to set up later following this same pattern.)

Send this request and you’ll get a response that looks like this:

{
  "id": "3cc79139-294a-449e-9366-46bc629ffddc",
  "tenantId": "myTenantName2157-xmcloudvani7a73-dev-2bda",
  "label": "OnUpdate Webhook Sandbox",
  "uri": "https://webhook.site/#!/view/d4ebda52-f7d8-4ae6-9ea2-968c40bc7f2f",
  "method": "POST",
  "headers": {
    "x-acme": "ContentUpdated"
  },
  "body": "",
  "createdBy": "CMS",
  "created": "2024-04-03T15:42:43.079003+00:00",
  "bodyInclude": null,
  "executionMode": "OnUpdate"
}

Try your GET request again on https://edge.sitecorecloud.io/api/admin/v1/webhooks, and you should see your webhook returned in the json response.

Try making some content updates and publishing from XM Cloud. Over at webhook.site, wait a few minutes and make sure you’re getting the json payload sent over. If so, you’ve set this up correctly.

To delete this webhook, you can send a DELETE request to https://edge.sitecorecloud.io/api/admin/v1/webhooks/<your-webhook-id>. Make sure you include your auth bearer token!

In the next post, we’ll go over handling this webhook to push content updates into our search index.

.NET not mapping fields with spaces in the headless SDK

TLDR: Don’t put spaces in your field names for Sitecore headless builds.

Quick post about a bug I uncovered today working on an XM Cloud project with a .NET rendering host.

We’re seeing inconsistent behavior when mapping layout service field data to types in .NET. In short, fields with spaces in the name sometimes do not deserialize into .NET types when you attempt to do so with the SDK.

Consider a page with 2 fields: Topic and Content Type. In our code, we have a class that looks like this:

namespace MySite.Models.Foundation.Base
{
  public class PageBase
  {
    [SitecoreComponentField(Name = "Content Type")]
    public ItemLinkField<Enumeration>? ContentType { get; set; }

    [SitecoreComponentField(Name = "Topic")]
    public ContentListField<Enumeration>? Topic { get; set; }
  }
}

When I create a component as a ModelBoundView,
.AddModelBoundView<PageBase>("PageHeader")
the fields map properly, and I get data in ContentType and Topic properties of my model.

When I try to map it from a service, like so:

namespace MySite.Services.SEO
{
  public class MetadataService
  {
    private SitecoreLayoutResponse? _response;

    public PageMetadata GetPageMetadata(SitecoreLayoutResponse? response)
    {
      _response = response;
      PageBase pageBase= new PageBase ();
      var sitecoreData = _response.Content.Sitecore;

      sitecoreData.Route?.TryReadFields<PageBase>(out pageBase);
      return pageMetadata;
    }
  }
}

I get no data in the Content Type field but I do in the Topic field. If I rename Content Type to ContentType, the field data is bound to the ContentType property as expected.

I dug into the code a little bit and it seems that the HandleReadFields method on the Sitecore.LayoutService.Client.Response.Model.FieldsReader is ignoring the property attributes: [SitecoreComponentField(Name = "Content Type")]
Instead it is just using the property name, which of course has no spaces in it because it’s a C# identifier.

Until this bug is corrected, the workaround is to rename your fields to not have spaces in them.