Webhooks
When processing monolithic datasets exceeding 1,000,000 rows, blocking HTTP requests will result in connection timeouts. Marigold heavily advocates for asynchronous webhook architectures.
Asynchronous Processing Flow
Instead of holding a TCP connection open while the Fellegi-Sunter algorithms iterate across millions of vector matrices, you can pass a callback_url in your initial payload. The Marigold API will immediately return a 202 Accepted containing a Job ID.
Once the analytical cluster finishes calculating the Z-scores and encrypting the resultant anomaly flags, we will execute an HTTP POST request against your callback URL containing the final AES-GCM encrypted payload.
Webhook Idempotency
Marigold guarantees At-Least-Once delivery. Your webhook ingestion controller must be idempotent. Use the session_fingerprint to ensure your database does not process the same callback payload multiple times in the event of a network retry.
HMAC-SHA256 Signature Verification
Because your webhook endpoint must be publicly accessible on the internet, you must cryptographically verify that incoming payloads actually originated from Marigold Insights, rather than a malicious actor attempting a spoofing attack.
Every webhook dispatched by Marigold includes a X-Marigold-Signature HTTP header. This signature is an HMAC-SHA256 hash of the raw request body, signed using your Webhook Secret.
1// Express.js Webhook Verification Middleware2import crypto from 'node:crypto';3import express from 'express';45const app = express();6const WEBHOOK_SECRET = process.env.MARIGOLD_WEBHOOK_SECRET; // Starts with whsec_78// WARNING: You must use the RAW body for HMAC verification.9// Do not use express.json() before verification.10app.post('/webhooks/marigold', express.raw({ type: 'application/json' }), (req, res) => {11 const signatureHeader = req.headers['x-marigold-signature'];1213 if (!signatureHeader) {14 return res.status(401).send('Missing Signature');15 }1617 // Generate our own signature based on the raw body18 const expectedSignature = crypto19 .createHmac('sha256', WEBHOOK_SECRET)20 .update(req.body)21 .digest('hex');2223 // Use timingSafeEqual to prevent timing attacks24 const isValid = crypto.timingSafeEqual(25 Buffer.from(signatureHeader),26 Buffer.from(expectedSignature)27 );2829 if (!isValid) {30 return res.status(401).send('Invalid Signature');31 }3233 // Parse the body now that it is validated34 const payload = JSON.parse(req.body.toString());3536 // Acknowledge receipt immediately to prevent Marigold from retrying37 res.status(200).send('OK');3839 // Process the payload asynchronously...40 processEncryptedVault(payload);41});