ALZ Monitoring Alerts Process
This page is a guide for ALZ Engineers to review and act on the Azure monitoring alerts within Azure Landing zone teams channel. Alerts are only generated for the Spokes we support end-to-end: Prod Hub, Prod Shared Services and Prod Small Services Space (aka SMSS or 3S).
ALZ Monitoring Rota
As an ALZ engineer you will use the current OOH rota (provided by service level management) as a guide for who will monitor the Azure Monitoring alerts channel in the day time. Please note the rota needs to be updated:
- When Providing Holiday or Sickness cover
- When we have new starter\leaver
- When it’s start of a new year
Below is a snapshot of the weekly rota:
| Week 1 | Week 2 | Week 3 | Week 4 |
|---|---|---|---|
| Alan | Ravi | Noor | Kokulan |
| 06/01/2025 | 13/01/2025 | 20/01/2025 | 27/01/2025 |
| 03/02/2025 | 10/02/2025 | 24/02/2025 | 17/02/2025 |
| 03/03/2025 | 10/03/2025 | 17/03/2025 | 24/03/2025 |
| 31/03/2025 | 14/04/2025 | 07/04/2025 | 21/04/2025 |
| 28/04/2025 | 05/05/2025 | 12/05/2025 | 19/05/2025 |
| 26/05/2025 | 02/06/2025 | 09/06/2025 | 16/06/2025 |
| 23/06/2025 | 30/06/2025 | 07/07/2025 | 14/07/2025 |
| 21/07/2025 | 28/07/2025 | 04/08/2025 | 11/08/2025 |
| 18/08/2025 | 25/08/2025 | 01/09/2025 | 15/09/2025 |
| 08/09/2025 | 22/09/2025 | 29/09/2025 | 06/10/2025 |
| 13/10/2025 | 20/10/2025 | 27/10/2025 | 03/11/2025 |
| 10/11/2025 | 17/11/2025 | 24/11/2025 | 01/12/2025 |
| 08/12/2025 | 15/12/2025 | 22/12/2025 | 29/12/2025 |
| 05/01/2026 | 12/01/2026 | 19/01/2026 | 26/01/2026 |
Alert Notifications
We have 2 channels under the Azure Landing Zone team in Microsoft Teams that are used to receive alert notifications.
Azure Monitor has been configured to generate alerts within the Monitoring Alerts channel.
The Notifications channel is used to receive information from 2 pipelines which run daily (“Routes missing from Hub Internal Subnet UDRs” and “Pipelines with build on fork enabled”) plus the monthly Azure Advisor recommendations.
Teams channels:
Example Monitoring Alert:
Reviewing Alert Notifications
ALZ Engineers will be responsible for reviewing the Alerts within the Monitoring Alerts and Notifications channels each day.
When reviewing the alert the engineer needs to determine whether it’s an Incident or not:
- A guide to our Incident Types.
- If we think it’s an Incident please follow the Incident Management Process.
If the Alert is not severe enough to be an incident we should at least follow Moderate Incident type as guide to a less involved process.
All alerts must be acknowledged by an ALZ Engineer, regardless whether it is an incident or not, to show that the alert as been reviewed and then resolved. This is done by replying to the Alert message that is created in the Teams chat.
Instructions to resolve: “Pipelines with build on fork enabled”
The pipeline is build-on-fork-audit.yml and details of the Shell script can be found at fork_trigger_checksh
- Find the pipeline in Azure DevOps
- Click
Edit(top right) - Click the three dots (top right)
- Click
Triggers - Click the Pull request validation entry
- Untick: Build pull requests from forks of this repository (see screenshot below)
- Click the dropdown next to “Save & queue” and select
Save
ALZ Incident Management
ALZ Incident Management Approach