Handling Vericlock Webhooks: Overcoming Challenges and Optimizing Costs

Integrating third-party services can sometimes feel like navigating a minefield. Recently, I encountered a tricky situation while integrating Vericlock webhooks at Prestige Lock and Door. Vericlock’s time-tracking service sent us JSON data containing nested and malformed fields, causing our webhook processing to fail. The primary issue was the unexpected data format and the lack of useful information from the Vericlock API Documentation. Here’s a breakdown of the problem and how I resolved it.

The Problem

Initially, Vericlock sent data with the Content-Type: application/x-www-form-urlencoded instead of application/json. This discrepancy made it difficult to parse the data correctly. Even popular platforms like Make couldn’t handle the format properly. Only Zapier managed to process it without issues.

Additionally, the data included complex nested structures, especially within the customFields section. This section was causing parsing errors because it contained another JSON object that wasn’t properly closed. Here’s an example of the problematic data:

{
  "guid":"[REDACTED]",
  "rootGuid":"[REDACTED]",
  "employeeGuid":"[REDACTED]",
  "jobGuid":null,
  "jobCode":null,
  "jobName":null,
  "serviceItemGuid":null,
  "serviceItemCode":null,
  "serviceItemName":null,
  "serviceRatePennies":0,
  "start":"2024-07-15T21:40:00.000Z",
  "end":null,
  "duration":0,
  "clockInReportId":16913206,
  "inDetails":{
    "method":"api",
    "report":{
      "type":"text",
      "value":"1"
    },
    "ipAddress":"[REDACTED]",
    "geoTagging":{
      "error":"DeviceTimeout",
      "message":"A timeout error occurred on the device trying to retrieve GPS coordinates"
    },
    "customFields":":{
      "guid":"[REDACTED]",
      "name":"Vehicle",
      "type":"list",
      "listItemGuid":"[REDACTED]",
      "listItemName":"none",
      "listItemValue":4
    }:{
      "displayName":"Regular",
      "minutes":0,
      "timeTypeId":1
    },{
      "displayName":"Overtime 1",
      "minutes":0,
      "timeTypeId":2
    },{
      "displayName":"Overtime 2",
      "minutes":0,
      "timeTypeId":3
    }:""
  }
}

Notice the customFields section? It contained another JSON object that was not properly closed, leading to parsing errors.

The Solution

To handle this, I devised a solution that involved sanitizing the incoming JSON string before parsing it. Here’s how I did it:

1. Extract the Raw JSON String: The incoming data was stored as a string, which needed to be sanitized.

2. Remove the Problematic Section: By identifying the customFields segment, I stripped it from the string.

3. Parse the Cleaned String: Finally, the cleaned JSON string was parsed into a usable JSON object.

Here’s the code I used:

const express = require('express');
const bodyParser = require('body-parser');
const logger = require('./logger');

const app = express();
app.use(bodyParser.urlencoded({ extended: true }));

app.post('/webhook', async (req, res) => {
  const rawBody = req.body;
  logger.info(`Raw Body: ${JSON.stringify(rawBody)}`);

  const jsonString = Object.keys(rawBody)[0];
  const sanitizedString = jsonString.split(',"customFields":')[0] + "}}";
  const parsedData = JSON.parse(sanitizedString);
  logger.info(`Parsed Data: ${JSON.stringify(parsedData)}`);

  // Process the parsed data further as needed

  res.status(200).send('Webhook processed successfully');
});

const port = process.env.PORT || 3000;
app.listen(port, () => {
  logger.info(`Webhook receiver listening at http://localhost:${port}`);
});

Key Takeaways

1. Ask for Webhook Feature Activation: Ensure you contact Vericlock support to enable the webhook feature for your account.

2. Data Parsing Challenges: Be prepared to handle unexpected data formats. Proper sanitization and parsing are crucial.

3. Cost Optimization: By integrating this solution, Prestige Lock and Door can transition from using Zapier (which incurs costs) to our own server, leading to significant savings.

Conclusion

This experience underscored the importance of flexibility and creativity in handling third-party integrations. It also demonstrated how we can optimize costs and improve system management by leveraging our own infrastructure. If you’re facing similar challenges, I hope this guide provides the insights you need to navigate and resolve them effectively.. Sharing this experience, I hope to help other developers facing similar issues with third-party integrations.