Push alert events from your own system to Flashduty through standard protocols to achieve automated alert noise reduction.Flashduty has already adapted webhook protocols for most common alert systems. For these systems, you should first use the corresponding integration for simplicity. This integration provides a standard HTTP interface that requires your development for adaptation. The advantage is that you can push any alert events you want to handle through on-call.
Steps#
In Flashduty#
You can obtain an integration push URL through either of the following two methods:Using Dedicated Integration#
When you don't need to route alert events to different collaboration spaces, this method is preferred as it's simpler.Expand
1.
Enter the Flashduty console, select Collaboration Space, and enter the details page of a space
2.
Select the Integrations tab, click Add Integration, and enter the add integration page
3.
Select Standard Alert Event integration, click Save to generate a card
4.
Click the generated card to view the Push URL, copy it for later use, and you're done
Using Shared Integration#
When you need to route alert events to different collaboration spaces based on the alert event's Payload information, this method is preferred.Expand
1.
Enter the Flashduty console, select Integration Center => Alert Events to enter the integration selection page
2.
Select Standard Alert Event integration:Integration Name: Define a name for the current integration
3.
Configure the default route and select the corresponding channel (after the integration is created, you can go to Route
to configure more routing rules)
4.
Click Save and copy the newly generated push URL for later use
I. Request Description#
Request Method#
POST, Content-Type:"application/json"
Request Parameters:#
Field | Required | Type | Description |
---|
Content-Type | Yes | string | Fixed value: application/json |
Query Strings:#
Field | Required | Type | Description |
---|
integration_key | Yes | string | Integration key for access control. Obtained after adding integration. |
Payload:#
Field | Required | Type | Description |
---|
title_rule | Yes | string | Alert title, no more than 512 characters, will be truncated if exceeded.
Supports dynamic title generation based on alert content, see Customizing Incidents for generation rules. |
event_status | Yes | string | Alert status.
Enumerated values (case-sensitive): Critical, Warning, Info, Ok.
When specified as Ok, it means automatic recovery of the alert. |
alert_key | No | string | Alert identifier, used to update or automatically recover existing alerts.
You can customize this value, but it cannot exceed 255 characters. You can also rely on system auto-generation, this value will be returned in the response.
If you're reporting a recovery event, this value must exist. |
description | No | string | Alert description, no more than 2048 characters, will be truncated if exceeded. |
labels | No | map | Alert label collection, key is the label name, value is the label value:
1. Both key and value of labels are string type, case-sensitive. 2. Label key should not exceed 128 characters, following Prometheus label naming conventions. Value should not exceed 2048 characters, will be truncated if exceeded. 3. Maximum of 50 labels. See Label Content Reference in Best Practices.
Example: "resource": "171.26.23.22", "check": "api latency > 500ms" |
Response#
Field Name | Required | Type | Description |
---|
request_id | Yes | string | Request ID for trace tracking |
error | No | Error | Error description, returned only when an error occurs |
data | No | Data | Report information |
Field Name | Required | Type | Description |
---|
alert_key | No | string | Alert identifier, can be used to report recovery events. If you specified an alert_key when reporting the event, this value remains unchanged. Otherwise, it's automatically generated by the system. |
Field Name | Required | Type | Description |
---|
code | Yes | string | Error code, see Code for enumerated values |
message | No | string | Error description |
Error Code | HTTP Status | Description |
---|
InvalidParameter | 400 | Parameter error |
InvalidContentType | 400 | Content-Type not supported |
MethodNotAllowed | 400 | HTTP Method not supported |
Unauthorized | 401 | Login authentication failed |
AccessDenied | 403 | Permission authentication failed |
RequestTooFrequently | 429 | Request too frequent |
RouteNotFound | 404 | Request Method+Path not matched |
ResourceNotFound | 400 | Account hasn't purchased resources, please go to the cost center to place an order |
NoLicense | 400 | Account has insufficient subscription licenses, please upgrade or purchase subscription in the cost center |
InternalError | 500 | Internal or unknown error |
II. Request Example#
{
"request_id": "0ace00116215ab4ca0ec5244b8fc54b0",
"data": {
"alert_key": "9qJ798NJoXS4UMVB5SHsNj"
}
}
{
"request_id": "0ace00116215abc0ba4e52449bd305b0",
"error": {
"code": "InvalidParameter",
"message": "integration_key is not a valid one"
}
}
III. Best Practices #
1.
Send events to Flashduty when alert status changes
2.
When an alert recovers, send an event with status Ok to close the alert. Otherwise, the alert will remain open. If your alert system doesn't have recovery events, we recommend manually sending recovery events
3.
Labels are event descriptions, and label content should be as rich as possible (specified when sending, or generated through enrichment rules), such as:Alert source, like host, cluster, check, or metric
Alert ownership information, like team, owner
Alert category information, like class (api, db, net)
IV. FAQ#
Why haven't I received alerts in Flashduty?
In Flashduty#
1.
Check if the integration shows Latest Event Time? If not, it means Flashduty hasn't received the push, prioritize checking your system.
2.
If you're using Shared Integration, first confirm if you've configured Routing Rules. Without routing rules, the system will directly reject new pushes as there's no collaboration space to handle your alerts. In this case, simply configure routing rules to your desired space.
In Your System#
1.
Confirm that your request URL exactly matches the URL in the integration details.
2.
Confirm that your service can access the external domain api.flashcat.cloud. If not, you need to enable external network access for the server or specifically for Flashduty's domain.
3.
Print Flashduty service's response to check for clear information.
If you still can't find the root cause after these steps, please contact us with the request_id from the request response.Why was the push request successful but no new alerts or incidents were generated?
Flashduty uses a 2-layer noise reduction mechanism:1.
First, it checks for duplicate alert events. If your pushed event is identical to a previously pushed event, the new event will be discarded.
2.
If the new event's status and description match the status, title, and description of the last event of its corresponding alert, the new event will be discarded while updating the alert's attributes.
3.
The new event might be discarded due to matching exclusion, discard, suppression, or silence rules.
4.
When a new event triggers a new alert, the system enters the second layer of noise reduction check, determining if the new alert can be merged into an active incident. If possible, it will only merge into the existing incident without generating a new one.