A server without monitoring is not secure, only quiet. A full disk, a runaway process or a thousand failed logins show up only when customers call. Monitoring is also your only proof that you noticed an incident in time.
Metrics tell you something is wrong; logs tell you what. Four metrics are mandatory on any server: free disk space, available memory, CPU load and open connections. For a web application add the 5xx error rate and 95th-percentile latency. Among the logs, authentication shows brute force and unusual logins, nginx shows scans and exploitation attempts, the application shows exceptions, and auditd records commands run as root and changes to sensitive files. You read them locally with journalctl; the central system keeps them.
The first thing an intruder does is clean the logs. If the only copy sits on the compromised machine, the investigation ends before it begins, so Grafana Alloy forwards logs in real time to a machine where production holds only an ingestion channel. Alerts work the opposite way: the more of them there are, the more they get ignored. Define a service level objective, such as at least 999 of every 1,000 requests succeeding, and alert when you burn through it too fast. Disk under 10 % free, or a login from outside the VPN, is worth waking someone for.
OUG 155/2024, the NIS2 transposition approved by Law 124/2025, requires entities registered with DNSC to send an early warning within 24 hours of identifying a significant incident, a notification within 72 hours and a final report one month later. Without logs and metrics you can neither identify the incident nor write the report. GDPR Article 33 requires notifying ANSPDCP within 72 hours of learning about a breach, and Article 32 requires the ability to detect one. Logs contain IP addresses, which are personal data, so give them a defined retention period and limited access.
| Signal | Source | Typical alert threshold | Why it matters |
|---|---|---|---|
| Disk space | node_exporter | under 10–15 % free for 15 minutes | The database refuses writes and logs are lost |
| HTTP 5xx errors | nginx, application | over 1 % of requests for 5 minutes | A direct symptom for users |
| p95 latency | application, nginx | above your defined SLO | Degradation before an outage |
| SSH logins | auth.log / journal | a successful login from outside the VPN | Brute force, compromised account |
| Sensitive files | auditd | any change in /etc/ssh or /etc/sudoers.d | Attacker persistence |
| TLS certificate | blackbox_exporter | expires in under 14 days | The site becomes unreachable on expiry |
journalctl --since "-1h" -p warning
journalctl -u ssh --since today | grep -E "Accepted|Failed"
sudo ausearch -m EXECVE -ts recent// /etc/alloy/config.alloy
loki.write "central" {
endpoint { url = "http://10.0.0.20:3100/loki/api/v1/push" }
}
local.file_match "system" {
path_targets = [
{__path__ = "/var/log/auth.log", job = "varlogs", host = "web-01"},
{__path__ = "/var/log/nginx/*.log", job = "varlogs", host = "web-01"},
]
}
loki.source.file "system" {
targets = local.file_match.system.targets
forward_to = [loki.write.central.receiver]
}groups:
- name: server
rules:
- alert: DiskAlmostFull
expr: node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} < 0.10
for: 15m
labels:
severity: critical
annotations:
summary: "Under 10 % free space on / ({{ $labels.instance }})"A night of aggressive scanning filled the disk with nginx logs and PostgreSQL began refusing writes. Nobody knew until the first complaint at breakfast. An alert at 15 % free space and a correct logrotate setup removed that whole class of incident.
On one compromised server the authentication logs had been wiped, so nobody knew how long the intruder had been inside. The copy in Loki, on a separate machine, showed the first successful login three weeks earlier, using an SSH key added by a former contractor.
Keep security logs, meaning authentication, sudo and auditd, for at least six months so you can investigate an incident found late. Web access logs contain IP addresses, which are personal data, so keep them for a period set in your GDPR policy, often 30–90 days, then aggregate or delete them.
For one server, an external uptime check plus simple disk and memory alerts can be enough at first. Prometheus and Loki become necessary once you run several machines or when you have to prove what happened during an incident.
Monitoring tracks system health: resources, errors, latency. A SIEM collects security events from several sources and correlates them, for example 500 failed logins followed by a successful one, then a new key in authorized_keys. On a small server, Loki with a few rules can cover both roles.
A production server starts getting attacked within minutes of receiving a public IP address, however small the project is. This guide ranks the controls by impact, from SSH access to incident response, with the Romanian rules that apply along the way. Every topic has its own page with a short answer, examples and ready-to-use configuration.