Webhook alerts - Zenskar Documentation

Concepts

What a webhook alert is

A webhook alert makes Zenskar send an HTTP POST to an endpoint you control whenever a chosen event occurs, such as a customer being created or an invoice being approved. Use it instead of polling the API for changes.

Events, categories, and subscriptions

Concept	Detail
Event name	A dotted string: category prefix plus a past-tense action, for example `invoice.approved`. Delivered in the body as `event_triggered`
Categories	Customer, Invoice, Contract, Payment, Entitlement, Usage event ingestion
Subscription	One webhook subscribes to any set of events, across any categories
Object reference	Every delivery names the affected object by ID and type; see App fields to payload keys

Webhook lifecycle

A webhook is Active or Paused; pausing keeps its configuration and deleting removes it. See Webhook states.

How a delivery is produced and sent

When a subscribed event occurs, the originating action (for example, approving an invoice) hands it to Zenskar’s delivery service and then returns. The hand-off is best-effort: if it fails, the originating action still succeeds. The delivery service matches the event against every enabled webhook subscribed to it, builds a signed request for each, and POSTs it to the endpoint over HTTPS. Transient failures are retried a few times; see Delivery attempts. Webhooks push changes as they happen. Use the Zenskar API to read current state or to backfill events you missed.

Delivery sequence

Delivery guarantees

Your webhook handler must be idempotent. Delivery is at-least-once and unordered, so deduplicate on event_id and apply events by occurred_at, not arrival order.

Guarantee	Behavior	Handle it by
Delivery	At-least-once. The delivery service skips a webhook whose delivery for this `event_id` already succeeded, and retries one that failed, but a duplicate can still arrive from a queue redelivery or from the same change being emitted twice	Recording each `event_id` you process and skipping repeats
Ordering	Not guaranteed; a slow delivery can overtake a faster one, so an older event can arrive after a newer one for the same object	Tracking the newest `occurred_at` applied per `object_id` and skipping any event at or before it, comparing only within the same `object_id`
Payload	A full snapshot of the object at `occurred_at` under `event_info`, not a diff, so a stale event overwrites newer values	Using `occurred_at` as the version

For example, two customer.updated events applied in arrival order rather than by occurred_at leave you with the older values.

Endpoint safety

The endpoint URL must be HTTPS. The delivery service rejects a URL that is not HTTPS, that uses a raw private or reserved IP address, that is localhost, that ends in .local, .internal, or .localhost, or that is metadata.google.internal. URL structure is checked when you save the webhook. On every delivery the delivery service also resolves the hostname and rejects it if any resolved address is in a private or reserved range.

Signed deliveries

Every request carries an X-Signature header: sha256= followed by the lowercase hex HMAC-SHA256 of the exact request body, keyed with the webhook’s secret. The body is compact JSON: a space follows each : and ,, non-ASCII characters are escaped, and keys are in insertion order, not sorted. Recompute the signature over the bytes as received and compare in constant time before trusting the payload. A match proves the request came from Zenskar unaltered.

Events not shown in the form

invoice.cancelled is produced internally when a full credit note with a cancel intent cancels an invoice, but the delivery service has no mapping for it, so it is not delivered to webhook endpoints and has no checkbox. entitlement.expired is defined but not currently emitted.

Effective values in payloads

One payload field is normalized rather than copied raw. A contract’s status is delivered as its effective status: a contract past its end date reports expired even if its stored status is still active. Every other field is copied as stored. See App fields to payload keys for the field map.

How-to guides

Open webhook alerts

Click the account menu at the bottom of the sidebar, and select Settings.
Open the Webhook Alerts tab.

If the tab reports that you are not authorized, ask an administrator to grant your role the webhook permissions. See Permissions.

Create a webhook

On Settings > Webhook Alerts, click Add webhook alert.
Enter a Webhook Name.
Optional: click + Add description and enter a description.
Enter the Endpoint URL. It must be an HTTPS URL that is reachable from the public internet and accepts an HTTP POST. A URL that is not HTTPS, or that points at a private or internal address, is rejected.
Enter a Secret Key. Use a long, random value, and store it where your endpoint can read it.
Leave Enable Webhook Alerts checked to start delivering immediately, or uncheck it to create the webhook paused.
Select at least one event (see Choose which events to subscribe to).
Click Create.

Events are grouped by category. Expand a category and check events such as Customer Created or Invoice Approved; the count beside each category shows how many are selected. At least one event is required to save.

Edit a webhook

On Settings > Webhook Alerts, open the actions menu on the webhook’s row, and select Edit.
Change any field. The secret key is masked; enter a new value only to replace it.
Click Update.

Pause or resume a webhook

Open the actions menu on the webhook’s row, and select Pause or Resume. A paused webhook delivers nothing until resumed.

Delete a webhook

On Settings > Webhook Alerts, open the actions menu on the webhook’s row, and select Delete.
Confirm.

Inspect delivery history

On Settings > Webhook Alerts, click the webhook’s row.
The webhook page shows its Endpoint, current Status, and the Tracking Events it subscribes to.
Alerts History lists events delivered to this webhook, most recently attempted first, each marked Succeeded or Failed. Each entry is one event; retries and manual resends update the same entry in place rather than adding a new one.
Select a delivery for Triggered Webhook Details (webhook ID, delivery status, last-updated time) and Event Information (the JSON body sent). A copy action copies the payload.

Resend a delivery

Open a delivery in Alerts History, then click Resend in Triggered Webhook Details. Zenskar re-sends the same event; only attempt_count changes, which also changes the signature.

Verify a signature

Compute the HMAC-SHA256 of the raw request body with the webhook’s secret, then compare it to the X-Signature header value after the sha256= prefix. Hash the bytes as received. Do not parse and re-serialize the JSON first: that changes the bytes and the hash.

import hashlib
import hmac

from fastapi import HTTPException


def verify_signature(payload_body: bytes, secret_token: str, signature_header: str):
    """Verify that the payload was sent from Zenskar.

    Args:
        payload_body: the raw request body, exactly as received (request.body())
        secret_token: the webhook's secret key
        signature_header: the value of the X-Signature header
    """
    if not signature_header:
        raise HTTPException(status_code=403, detail="X-Signature header is missing")
    digest = hmac.new(
        secret_token.encode("utf-8"), msg=payload_body, digestmod=hashlib.sha256
    )
    expected = "sha256=" + digest.hexdigest()
    if not hmac.compare_digest(expected, signature_header):
        raise HTTPException(status_code=403, detail="Signatures did not match")

Make your endpoint idempotent

A delivery can arrive more than once. event_id is unique per event: record the ones you have processed and skip repeats before doing work.

Apply events in timestamp order, not arrival order

Events can arrive out of order, and object payloads are full snapshots, so applying a late older event overwrites newer values. Track the newest occurred_at applied per object_id and skip anything older or equal:

from datetime import datetime

occurred_at = datetime.fromisoformat(event["occurred_at"])
last = store.get(event["object_id"])
if last is not None and occurred_at <= last:
    return  # stale: a newer state for this object is already applied

apply(event["event_info"])
store.set(event["object_id"], occurred_at)

Compare occurred_at only within the same object_id. Order by the top-level occurred_at. Some payloads also carry an updated_at inside event_info, but it is not present on every event and can differ slightly from occurred_at, so prefer the top-level field.

Respond correctly

Return HTTP 200, 201, 202, or 204 as soon as you have stored or queued the event, and process it asynchronously. Respond within 30 seconds. Any other status, a timeout, or a connection error marks the delivery Failed. Transient failures are retried; a 4xx response is treated as permanent and is not retried. See Delivery attempts.

Troubleshoot deliveries

Symptom	Cause and fix
A delivery shows Failed	The endpoint did not return 200, 201, 202, or 204 within 30 seconds, or was unreachable. Confirm it is reachable from the public internet over HTTPS, returns a success status, and responds in time. A 4xx response is not retried.
The signature never matches	Hash the raw body bytes, not a parsed-and-reserialized copy. The body is compact JSON with keys in insertion order; re-serializing changes bytes such as key order and whitespace, which changes the hash. Confirm you are using the secret key configured on that webhook.
A synced value reverted to an earlier value, or an event arrived twice	Expected: events are unordered and each payload is a full snapshot, so arrival-order application can regress a field when a slow delivery overtakes a faster one. Deduplicate on `event_id` and apply by `occurred_at`. See Apply events in timestamp order, not arrival order.
Saving the webhook fails with a URL error	The endpoint URL must be HTTPS and must not target `localhost`, a private or reserved IP address, or a `.local`, `.internal`, or `.localhost` host.
`invoice.cancelled` or `entitlement.expired` cannot be selected	Neither is in the form. `invoice.cancelled` is also not delivered to endpoints; `entitlement.expired` is not currently emitted.

Reference

Location

Settings > Webhook Alerts.

Permissions

Action	Permission
Open the tab, view webhooks and delivery history	`can_read_webhook`
Create a webhook	`can_create_webhook`
Edit, pause, or resume a webhook	`can_update_webhook`
Delete a webhook	`can_delete_webhook`
Resend a delivery	`can_trigger_webhook`

Create and delete also require can_read_webhook. If the tab reports that you are not authorized, ask an administrator to grant these to your role.

Webhook list columns

The table on Settings > Webhook Alerts lists your webhooks with these columns:

Column	Description
Webhook Name	The name given to the webhook
Status	Active or Paused
Created At	When the webhook was created

Click a row to open the webhook and its delivery history.

Webhook form fields

Field	Required	Notes
Webhook Name	Yes	Free-text label
Description	No	Revealed by the + Add description action
Endpoint URL	Yes	Must be an HTTPS URL; rejected if it is not HTTPS or targets a private or internal address. Receives each delivery as an HTTP POST
Secret Key	Yes	Signs every delivery; masked when editing, enter a value only to replace it
Enable Webhook Alerts	No	Checked by default; uncheck to create the webhook paused
Events	Yes	At least one event must be selected

Webhook states

State	Meaning
Active	Matching events are delivered
Paused	No deliveries; the endpoint, secret, and event selection are kept. Resume to restart

Deleting removes the webhook; it is not a state.

Form and payload names

Form category	Event prefix	`object_type`
Customer	`customer.`	`customer`
Invoice	`invoice.`	`invoice`
Contract	`contract.`	`contract`
Payment	`payment.`	`payment`
Entitlement	`entitlement.`	`entitlement`; `entitlement_block` for `entitlement.granted`
Usage Event Ingestion (S3)	`ingestion.`	`raw_metric`

Each checkbox is labelled <Category> <Action>, for example Payment Refunded for payment.refunded.

Event catalog

These are the event_triggered values delivered in the payload. In the webhook form the same events appear as checkboxes labelled by category and action, for example invoice.approved as Invoice Approved. invoice.cancelled and entitlement.expired have no checkbox, and invoice.cancelled is not delivered to endpoints.

Customer

Event	Form label	Delivered when
`customer.created`	Customer Created	A customer is created
`customer.updated`	Customer Updated	A customer’s details, addresses, or configuration change

Invoice

Event	Form label	Delivered when
`invoice.created`	Invoice Created	An invoice is created
`invoice.updated`	Invoice Updated	Invoice fields change, or a payment is recorded or reversed against it
`invoice.deleted`	Invoice Deleted	A draft or upcoming invoice is deleted
`invoice.approved`	Invoice Approved	An invoice is approved
`invoice.voided`	Invoice Voided	A full credit note voids the invoice
`invoice.cancelled`	Not in form	Produced internally when a full credit note with a cancel intent cancels the invoice, but not delivered to webhook endpoints
`invoice.regenerated`	Invoice Regenerated	An invoice is regenerated after a contract edit

Contract

Event	Form label	Delivered when
`contract.created`	Contract Created	A contract is created
`contract.updated`	Contract Updated	A contract, phase, or price changes; also on pause, pause edit, and resume
`contract.activated`	Contract Activated	A contract moves from a non-active state to an effective active state
`contract.deleted`	Contract Deleted	A contract is deleted

Payment

Event	Form label	Delivered when
`payment.created`	Payment Created	A payment record is created
`payment.updated`	Payment Updated	A payment changes, including the original payment after a refund
`payment.succeeded`	Payment Succeeded	A payment provider confirms the payment succeeded
`payment.failed`	Payment Failed	A payment attempt fails: a provider reports failure, a card is declined, or a 3-D Secure challenge is not completed
`payment.refunded`	Payment Refunded	A refund is created against a payment

Entitlement

Event	Form label	Delivered when
`entitlement.granted`	Entitlement Granted	An entitlement block is granted to a customer
`entitlement.exhausted`	Entitlement Exhausted	A customer’s remaining balance for an entitlement reaches zero or below after usage is committed
`entitlement.expired`	Not in form	Reserved; not currently emitted

Usage Event Ingestion

Shown in the form as Usage Event Ingestion (S3).

Event	Form label	Delivered when
`ingestion.completed`	Ingestion Completed	An uploaded usage-data file finishes ingesting, fully or partially
`ingestion.failed`	Ingestion Failed	An uploaded usage-data file fails to ingest

Delivery request

Part	Value
Method	`POST`
`Content-Type`	`application/json`
`X-Signature`	`sha256=` followed by the HMAC-SHA256 of the raw body, keyed with the secret key

Zenskar populates each delivery body with these fields, in this order:

Field	Description
`event_triggered`	The dotted event name, for example `invoice.approved`
`object_type`	The object’s type, for example `invoice`; `entitlement_block` for `entitlement.granted`, `raw_metric` for `ingestion.*`
`occurred_at`	ISO 8601 timestamp of the change, `T` separated; order events per `object_id` by this
`event_id`	Unique identifier for this event; use it to deduplicate
`object_id`	Identifier of the affected object
`event_info`	Full snapshot of the object at `occurred_at`, not a diff. For `ingestion.*` it is the ingestion result. Timestamps inside an object snapshot use a space separator instead of `T`
`attempt_count`	Which delivery attempt this POST is, starting at 1; increments on each retry and each manual resend

The body carries no organization identifier. Use a separate endpoint or secret per organization, or read the organization ID from event_info (organisation for customer, organisation_id for invoice and payment, organization_id for contract; entitlement and ingestion payloads carry none).

App fields to payload keys

event_info is a full snapshot of the object, minus internal and sensitive fields (raw billing data, internal identifiers). Each table maps the field as shown in the Zenskar app to its key inside event_info. A blank App field means the value has no single field in the app. For what each field means, see the module docs: Customers, Life cycle of an invoice, Contract, Payments, Entitlements.

Customer

App field	Payload key	Notes
Customer name	`customer_name`
External ID	`external_id`	Your identifier for the customer
Email	`email`
Phone number	`phone_number`
Billing address	`address`	Object with address lines, city, state, zip, country
Shipping address	`ship_to_address`
Tax ID	`tax_info`	List of tax registrations
Business entity	`business_entity_id`
Custom attributes	`custom_attributes`	Module custom attributes
	`custom_data`	Free-form key-value data
Charge invoices	`auto_charge_enabled`
Email communications	`communications_enabled`
	`id`	Zenskar customer ID (UUID)
	`organisation`	Organization ID
	`created_at`, `updated_at`, `deleted_at`	Timestamps; `deleted_at` is set when the customer is deleted

Invoice

App field	Payload key	Notes
Invoice number	`invoice_number`
Status	`status`
Customer	`customer_id`	`payer_customer_id` when the payer differs
Invoice date	`invoice_date`
Due date	`due_date`
Currency	`currency`
Total	`invoice_total`
Amount due	`amount_due`
Billing period	`period_begin_date`, `period_end_date`
Custom attributes	`custom_attributes`
Payment link	`payment_url`
	`id`	Invoice UUID
	`external_id`	External identifier
	`organisation_id`, `contract_id`	Organization and source contract
	`invoice_type`	`scheduled` or `advance_payment`
	`applied_balance`, `invoice_pdf`	Wallet balance applied; PDF URL
	`promise_due_date`, `bill_for_date`	Promised payment date; bill-for date
	`payment_term_type`, `due_period`, `payment_terms_config`	Payment terms
	`include_payment_link`, `partial_payment_enabled`, `auto_charge_enabled`, `use_wallet_balance`	Payment behavior flags
	`approved_at`, `paid_at`, `sent_at`, `created_at`, `updated_at`, `deleted_at`	Timestamps

Contract

App field	Payload key	Notes
Name	`name`
Description	`description`	Internal description
Status	`status`	Effective status: `draft`, `active`, `paused`, `expired`, `disputed`
Customer	`customer_id`	`invoice_payer_customer_id` when the payer differs
Currency	`currency`
Contract period	`start_date`, `end_date`
Start billing cycle on	`anchor_date`
Renewal policy	`renewal_policy`
Contract link	`contract_link`	URL to the signed agreement
Tags	`tags`
Custom attributes	`custom_attributes`
	`id`	Contract UUID
	`organization_id`
	`is_last_day_of_month`, `bill_parent_customer`	Billing configuration
	`created_at`, `updated_at`, `deleted_at`	Timestamps

Payment

App field	Payload key	Notes
Type	`type`	`payment`, `refund`, or `tax_withheld`
Status	`status`
Payment method	`payment_method`
Amount	`amount`	Smallest currency unit
Currency	`currency_code`
Payment date	`timestamp`	Unix epoch
Transaction ID	`external_id`
Customer	`customer_id`
Invoices	`invoice_ids`	Invoices the payment is applied to
	`id`	Payment UUID
	`receipt_number`
	`organisation_id`, `connector_id`, `connector_name`	Organization and gateway
	`parent_id`	Original payment, for a refund
	`payment_link_id`
	`amount_refunded`
	`autocharge`	Whether the payment was auto-charged
	`payment_method_details`, `balance_transactions`	Gateway detail
	`error_text`	Error description for a failed payment
	`custom_attributes`
	`created_at`, `updated_at`, `deleted_at`	Timestamps

Entitlement

Entitlement payloads have no app form fields and carry no organization ID.

Payload key	Meaning
`id`	Entitlement-customer record ID
`entitlement_id`	Entitlement definition ID
`customer_id`	Customer ID
`quantity`, `quantity_used`	Granted and used amounts
`active_from`, `expiry_at`	Validity window
`contract_id`, `invoice_id`, `product_id`	What granted the entitlement

For entitlement.granted, event_info is the full entitlement-customer record and object_id is that record’s ID. For entitlement.exhausted, event_info carries only entitlement_id and customer_id, every other key is null, and object_id is the entitlement definition ID.

Usage event ingestion

Sent when a usage-data file finishes ingesting. It has no form; the payload mirrors the ingestion result and carries no organization ID.

Payload key	Meaning
`raw_metric_id`, `raw_metric_name`	The raw metric the file was ingested into
`s3_key`	S3 key of the ingested file
`outcome`	`success`, `partial`, `failed`, or `dlq_escalated`
`total_rows`	Rows in the file
`valid_rows`, `invalid_rows`	Rows that passed or failed schema validation
`success_rows`, `failure_rows`	Rows that landed or did not
`error_file_s3_path`	S3 path to the error-summary file, if any
`completed_on`	Completion time (ISO 8601 UTC)
`failure_reason`	Machine reason code on failure
`trace_id`	Per-file idempotency and correlation key
`message`	Human-readable summary

Delivery outcome

Status	Meaning
Succeeded	The endpoint returned HTTP 200, 201, 202, or 204
Failed	The endpoint returned any other status, timed out, or was unreachable

Delivery attempts

Property	Behavior
Attempts	Up to 3: the first delivery plus 2 automatic retries
Backoff	Exponential, capped at 600 seconds between attempts; in practice the retries land within seconds
Timeout	30 seconds per attempt
Retried	Endpoint 5xx, connection refused or closed, connection failure, TLS error, timeout
Not retried	Any endpoint 4xx, `ssrf_blocked`, `dns_error`, `too_many_redirects`, `configuration_error`
Deduplication	Before sending, the delivery service skips a webhook whose delivery for this `event_id` already succeeded, and retries in place one that failed
Manual retry	Resend on any delivery in Alerts History; increments `attempt_count`

Delivery error codes

Shown on a failed delivery.

Code	Meaning	Retried
`timeout`	No response within 30 seconds	Yes
`connection_refused`	The endpoint refused the connection	Yes
`connection_closed`	The endpoint closed the connection early	Yes
`connection_failed`	The connection could not be established	Yes
`ssl_error`	TLS handshake failed	Yes
`proxy_error`	Delivery-service proxy error	Yes
`internal_error`	Unexpected error in the delivery service	Yes
`dns_error`	The hostname did not resolve	No
`ssrf_blocked`	The URL failed the private or internal address check	No
`too_many_redirects`	The endpoint redirected too many times	No
`configuration_error`	The webhook is misconfigured, for example a missing secret	No

Signature

Detail	Value
Header	`X-Signature`
Algorithm	HMAC-SHA256
Key	The webhook’s secret key
Signed content	The exact request body bytes
Body format	Compact JSON: a space after each `:` and `,`, non-ASCII escaped, keys in insertion order, not sorted
Format	`sha256=` followed by the lowercase hex digest

Delivery semantics

Property	Behavior
Emission	Best-effort; a webhook failure never blocks or reverses the action that produced the event
Delivery	At-least-once. A successful delivery is not repeated for the same `event_id` and webhook, but duplicates are still possible. Deduplicate on `event_id`
Ordering	Not guaranteed, for a single object or overall; a slow delivery can overtake a faster one. Apply events by `occurred_at`, not arrival order
Payload	A full snapshot of the object at `occurred_at` under `event_info`, not a diff
Retries	Up to 2 automatic retries for transient failures; see Delivery attempts
Manual retry	Resend on any delivery in Alerts History

​Concepts

​What a webhook alert is

​Events, categories, and subscriptions

​Webhook lifecycle

​How a delivery is produced and sent

​Delivery sequence

​Delivery guarantees

​Endpoint safety

​Signed deliveries

​Events not shown in the form

​Effective values in payloads

​How-to guides

​Open webhook alerts

​Create a webhook

​Choose which events to subscribe to

​Edit a webhook

​Pause or resume a webhook

​Delete a webhook

​Inspect delivery history

​Resend a delivery

​Verify a signature

​Make your endpoint idempotent

​Apply events in timestamp order, not arrival order

​Respond correctly

​Troubleshoot deliveries

​Reference

​Location

​Permissions

​Webhook list columns

​Webhook form fields

​Webhook states

​Form and payload names

​Event catalog

​Customer

​Invoice

​Contract

​Payment

​Entitlement

​Usage Event Ingestion

​Delivery request

​App fields to payload keys

​Customer

​Invoice

​Contract

​Payment

​Entitlement

​Usage event ingestion

​Delivery outcome

​Delivery attempts

​Delivery error codes

​Signature

​Delivery semantics

Concepts

What a webhook alert is

Events, categories, and subscriptions

Webhook lifecycle

How a delivery is produced and sent

Delivery sequence

Delivery guarantees

Endpoint safety

Signed deliveries

Events not shown in the form

Effective values in payloads

How-to guides

Open webhook alerts

Create a webhook

Choose which events to subscribe to

Edit a webhook

Pause or resume a webhook

Delete a webhook

Inspect delivery history

Resend a delivery

Verify a signature

Make your endpoint idempotent

Apply events in timestamp order, not arrival order

Respond correctly

Troubleshoot deliveries

Reference

Location

Permissions

Webhook list columns

Webhook form fields

Webhook states

Form and payload names

Event catalog

Customer

Invoice

Contract

Payment

Entitlement

Usage Event Ingestion

Delivery request

App fields to payload keys

Customer

Invoice

Contract

Payment

Entitlement

Usage event ingestion

Delivery outcome

Delivery attempts

Delivery error codes

Signature

Delivery semantics