I (Markus De Shon) attended a great talk by Sounil Yu: "The DIE Triad: The Past, Present, and Future of Security" (you can watch it on YouTube). He gives a sobering survey of the history of IT and Security since the 1980s, with the end result being: we can't prevent intrusions completely, we need to be able to Recover effectively, perhaps automatically. We need to be resilient or even antifragilewhere intrusions make us even more resilient.
The CIA Triad (the past)
To get there, he proposes that the past approach has been to have "pet" systems that we love and cultivate, patching and improving them over time but that they essentially persist over long periods of time. These systems require careful management under the C.I.A. triad (Confidentiality / Integrity / Availability) to survive.
The DIE Triad (present/future)
What he says we need to move to is "cattle" systems that are Distributed, Immutable and Ephemeral (forming the new "D.I.E. Triad"). So, think of systems deployed as Kubernetes containers, horizontally scaled and deployed across multiple regions and availability zones.
Each system is "immutable" in the sense that there's no expectation of being patched in place, updates are redeployments with a fresh image (where a blue/green failover, a.k.a. red/black failover, can ensure full availability during the upgrade). The underlying filesystem is (for example) a set of overlayfs layers that are readonly, with a final "copy on write" layer on top for any files that are changed.
Detection and FIM
One important component of such an approach is: how do we ensure robustness against intrusions?
On his DIE webpage Sounil outlines several test cases that are relevant, and the expected responses under DIE (in particular, for immutability):
Modify config file on production server (alert, revert)
Inject unauthorized binary (alert)
Change firewall rules (alert, revert)
Tamper with application code (alert)
Alter DB schema (alert)
Change system time or ntp source (alert)
Edit /etc/passwd (alert)
Modify logs (alert)
Deploy unapproved image (block)
All these manifest at the system level as changes to the filesystem, except:
Alter DB Schema happens within a database, which, if it manifests at the filesystem level at all, is a nontrivial deep change that can only be detected within the context of the database itself.
Deploy unapproved image is a pre-boot issue at the deployment level so it's out of scope for system-level analysis.
So, aside from those exceptions, I would argue that File Integrity Monitoring (FIM) can play an important role here.
Resilience and antifragility
I would also argue that to achieve resilience, we need the alert to trigger an immediate snapshot, and redeployment of the affected container. If the system is already distributed/ephemeral, there should be no problem with doing so.
To achieve antifragility, further analysis is needed to establish how the change was effected, and to close the hole. To extent that this analysis, fix and redeployment can be done automatically, the system becomes self-healing.
I think all of this is feasible with modern AI-driven system designs. There is risk here, but with the aforementioned blue/green failover which includes tests and quality metrics with automatic rollback on failure, I think these risks are manageable. As the song/saying goes: "If you liked it, you should have put a test on it."