005 - Use Log Aggregation Platform
Date: 2020-05-17
Status
✅ Accepted
Context
The PTTP programme requires that security logging is enabled to support the early stages of deployment. At this time the MoJ Operational Security Team (OST), does not have the immediate capacity to consume and process logs generated by the new devices and services being provisioned. We need an approach to aggregating logs for both traffic flow (Palo Alto) and the DNS/DHCP service for PTTP and to eventually stream or provide access to these logs for collection by OST.
Logging requirements for early beta
Areas of security logging capabilities
- Cloud
- Network appliances
Below is a list of the logs that maybe generated and delivered for each element.
Cloud
- IP connections (source and destination address)
- Security group flow logs (i.e. security group ingress/egress traffic logging)
- Login Success and failures
- Multi-factor authentication success and failures
- Logout events
- Group/role creation, modification or deletion
- Group role membership changes (addition/subtraction)
- Group role elevation (e.g. temporary assumed elevated privilege for finite time)
- Control-plane event logs (logging events generated by operations undertaken within the environment)
- OS security event logs (e.g. Windows VM event logs, Linux VM Syslog)
- CloudWatch logs (AWS)
- CloudTrail (AWS) #### Network appliances
- IP connections (source and destination address)
- Login Success and failure events
- Multi-factor authentication success and failures
- Logout events
- Power/service on/off
- Creation/registration and deletion/deregistration
- Software update events/status
- IP allocation/deallocation
- Firewall/routing rule creation, modification or deletion
- Network change events (e.g. additional or removal of virtual networks or interfaces)
- Successful/unsuccessful inbound service daemon connections
- Unsuccessful outbound connections where the network traffic is not associated to an inbound request
- Successful and unsuccessful DNS queries
- Infrastructure node/end-user device registration/de-registration (as applicable based on device type)
- IP address allocation/deallocation
- DHCP IP address request, IP address allocation, IP address lease duration, and IP address leased
- Unsuccessful client EUD IP address allocation, including requesting MAC address and the DHCP scope identifier
The following should not be included in log files:
- Secrets (e.g. passwords or session keys)
- Payment card details (e.g. related to PCI-DSS)
Decision
We will store all logs in S3 buckets for later ingestion by the Operational Security Team logging platform.
Update 2020-06-17 The Operational Security Logging Platform is ready to accept these logs and the decision was made to ship logs rather than store.
Consequences
Advantages
- We don’t need to stand up our own logging infrastructure
- Availability of logs from different sources in one location.
Disadvantages
- Reliant on another team which means we may need to wait sometime before we get an aggregated view of our logs.