Webhook Setup

Overview

Webhooks allow your content management systems and data sources to notify UDL when content changes. This enables near real-time updates without constant polling.

Webhook Architecture

┌─────────────────┐     Content Change     ┌─────────────────┐
│   Contentful    │ ────────────────────▶  │   UDL Server    │
│   Shopify       │     POST webhook       │                 │
│   Your CMS      │                        │   /_webhooks/   │
└─────────────────┘                        └────────┬────────┘
                                                    │
                                           Queue + Debounce
                                                    │
                                                    ▼
                                           ┌─────────────────┐
                                           │  Batch Process  │
                                           │  Update Nodes   │
                                           └─────────────────┘

UDL webhooks are:

• Queued: Rapid consecutive webhooks are batched together
• Debounced: Processing waits briefly for more webhooks to arrive
• Idempotent: Safe to receive duplicate webhooks

Webhook URL Format

All webhooks follow this URL pattern:

POST https://your-udl-server.com/_webhooks/{pluginName}/{path}

Examples

# Contentful default webhook
POST https://udl.your-domain.com/_webhooks/contentful/sync

# Custom plugin webhook
POST https://udl.your-domain.com/_webhooks/my-plugin/content-update

Default Webhook Handler

UDL provides a standardized webhook handler for CRUD operations. Plugins can opt into this handler for common use cases.

Payload Format

interface DefaultWebhookPayload {
  operation: 'create' | 'update' | 'delete' | 'upsert';
  nodeId: string;
  nodeType: string;
  data?: Record<string, unknown>;  // Required for create/update/upsert
}

Operations

Example Payloads

Create a new product:

{
  "operation": "create",
  "nodeId": "product-123",
  "nodeType": "Product",
  "data": {
    "title": "New Product",
    "price": 99.99,
    "slug": "new-product"
  }
}

Update existing product:

{
  "operation": "update",
  "nodeId": "product-123",
  "nodeType": "Product",
  "data": {
    "title": "Updated Product",
    "price": 79.99
  }
}

Delete a product:

{
  "operation": "delete",
  "nodeId": "product-123",
  "nodeType": "Product"
}

Upsert (create or update):

{
  "operation": "upsert",
  "nodeId": "product-456",
  "nodeType": "Product",
  "data": {
    "title": "Product",
    "price": 49.99
  }
}

Contentful Webhook Setup

Step 1: Get Your Webhook URL

Your Contentful webhook URL follows this pattern:

https://your-udl-server.com/_webhooks/contentful/sync

Step 2: Configure in Contentful

1. Go to Settings → Webhooks in Contentful
2. Click Add Webhook
3. Configure:

Step 3: Select Events

Under Triggers, select the events you want to sync:

Entry Events:

• Publish
• Unpublish
• Delete

Asset Events:

• Publish
• Unpublish
• Delete

Step 4: Configure Headers

Add any required headers:

Step 5: Test the Webhook

1. Click Save
2. Use the Test button to send a test payload
3. Check UDL logs for received webhook:

# Docker
docker logs udl

# systemd
sudo journalctl -u udl -f

# Railway CLI
railway logs -f

Shopify Webhook Setup

Step 1: Determine Webhook URL

https://your-udl-server.com/_webhooks/shopify/sync

Step 2: Configure via Admin

1. Go to Settings → Notifications in Shopify Admin
2. Scroll to Webhooks
3. Click Create webhook

Step 3: Select Events

Configure webhooks for these topics:

Step 4: Configure Each Webhook

For each event:

Custom Webhook Integration

For other data sources, implement a custom webhook handler:

1. Register Webhook in Plugin

import { defineConfig, WebhookRegistry } from 'universal-data-layer';

export const config = defineConfig({
  type: 'source',
  name: 'my-plugin',
});

export function onLoad({ webhookRegistry }: { webhookRegistry: WebhookRegistry }) {
  webhookRegistry.register({
    pluginName: 'my-plugin',
    path: 'content-update',
    handler: async (req, res, context) => {
      const { body, actions } = context;

      // Process the webhook payload
      await actions.createNode({
        internal: {
          id: body.id,
          type: 'MyContent',
          owner: 'my-plugin',
        },
        ...body.data,
      });

      res.writeHead(200);
      res.end(JSON.stringify({ success: true }));
    },
  });
}

2. Configure Source to Send Webhooks

Point your data source to:

POST https://your-udl-server.com/_webhooks/my-plugin/content-update

Webhook Security

Signature Verification

Verify webhook signatures to ensure requests come from legitimate sources:

webhookRegistry.register({
  pluginName: 'my-plugin',
  path: 'secure-webhook',
  verifySignature: async (req, rawBody) => {
    const signature = req.headers['x-webhook-signature'];
    const secret = process.env['WEBHOOK_SECRET'];

    if (!signature || !secret) {
      return false;
    }

    const crypto = await import('node:crypto');
    const expectedSignature = crypto
      .createHmac('sha256', secret)
      .update(rawBody)
      .digest('hex');

    return signature === expectedSignature;
  },
  handler: async (req, res, context) => {
    // Handle verified webhook
  },
});

IP Allowlisting

In production, consider restricting webhook endpoints to known IPs using your reverse proxy:

location /_webhooks/ {
    # Contentful IPs (example - check current IPs)
    allow 52.213.82.0/24;
    allow 52.213.83.0/24;
    deny all;

    proxy_pass http://udl_backend;
}

HTTPS Only

Always use HTTPS for webhook endpoints:

• Encrypts payload in transit
• Prevents man-in-the-middle attacks
• Required by most CMS platforms

Testing Webhooks

Using curl

Test your webhook endpoint manually:

# Test upsert operation
curl -X POST https://your-udl-server.com/_webhooks/contentful/sync \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "upsert",
    "nodeId": "test-123",
    "nodeType": "TestContent",
    "data": {
      "title": "Test Content",
      "body": "This is a test"
    }
  }'

Expected response:

{
  "queued": true
}

Verify Node Was Created

# Query for the created node
curl -X POST https://your-udl-server.com/graphql \
  -H "Content-Type: application/json" \
  -d '{
    "query": "{ testContent(id: \"test-123\") { title body } }"
  }'

Local Development Testing

For local testing without exposing your server:

Option 1: ngrok

# Expose local UDL to internet
ngrok http 4000

# Use the ngrok URL in your CMS webhook config
# https://abc123.ngrok.io/_webhooks/contentful/sync

Option 2: Webhook.site

1. Go to webhook.site
2. Configure CMS to send to webhook.site URL
3. Inspect payloads to understand format
4. Replay to local server with curl

Debugging

Check UDL Logs

Webhook activity is logged:

# Successful webhook
Webhook received: contentful/sync
✅ Default webhook [contentful]: Created node Product:product-123

# Failed webhook
⚠️ Default webhook handler [contentful]: Invalid payload received

Common Issues

404 Not Found

• Verify URL format: /_webhooks/{plugin}/{path}
• Check plugin is registered
• Verify webhook path matches registration

400 Invalid Payload

• Check JSON is valid
• Verify required fields: operation, nodeId, nodeType
• Include data for create/update/upsert operations

401 Invalid Signature

• Verify secret matches on both sides
• Check signature header name
• Ensure raw body is used for signature calculation

413 Payload Too Large

• UDL limits webhook bodies to 1MB
• For larger payloads, fetch data separately using nodeId

Webhook Queue Status

Webhooks are queued before processing. If updates seem delayed:

1. Check batch processing is running
2. Verify no errors in processing logs
3. Default debounce is 100ms - rapid webhooks are batched

Outbound Webhooks

UDL can notify external services when nodes change:

import { defineConfig, OutboundWebhookManager } from 'universal-data-layer';

const webhookManager = new OutboundWebhookManager({
  url: process.env['EXTERNAL_WEBHOOK_URL'],
  headers: {
    'Authorization': `Bearer ${process.env['WEBHOOK_TOKEN']}`,
  },
});

// Trigger when nodes are created/updated
webhookManager.notify({
  event: 'node.updated',
  nodeType: 'Product',
  nodeId: 'product-123',
});

Use cases:

• Trigger Vercel revalidation
• Notify search index to re-crawl
• Update external caches
• Trigger downstream builds

Next Steps

• Production Deployment - Run alongside your app
• Standalone Deployment - Run UDL as a separate server