…according to a Twitter post by the Chief Informational Security Officer of Grand Canyon Education.
So, does anyone else find it odd that the file that caused everything CrowdStrike to freak out, C-00000291-
00000000-00000032.sys was 42KB of blank/null values, while the replacement file C-00000291-00000000-
00000.033.sys was 35KB and looked like a normal, if not obfuscated sys/.conf file?
Also, apparently CrowdStrike had at least 5 hours to work on the problem between the time it was discovered and the time it was fixed.
The fact that a single bad file can cause a kernel panic like this tells you everything you need to know about using this kind of integrated security product. Crowdstrike is apparently a rootkit, and windows apparently has zero execution integrity.
I’m not sure why you think this statement is so profound.
CrowdStrike is expected to have kernel level access to operate correctly. Kernel level exceptions cause these types of errors.
Windows handles exceptions just fine when code is run in user space.
This is how nearly all computers operate.
This is a pretty hot take. A single bad file can topple pretty much any operating system depending on what the file is. That’s part of why it’s important to be able to detect file corruption in a mission critical system.
This was a binary configuration file of some sort though?
Something along the lines of:
Would have helped greatly here.
Edit: oh it’s more like an unsigned binary blob that gets downloaded and directly executed. What could possibly go wrong with that approach?
We agree, but they were responding to “windows apparently has zero execution integrity”.
Security products of this nature need to be tight with the kernel in order to actually be effective (and prevent actual rootkits).
That said, the old mantra of “with great power” comes to mind…
with great power, don’t lay off the testing team (force return to office or get fired ultimatums)
It’s fine, they’ve just switched to a crowd-sourced testing strategy.
Yeah pretty much all security products need kernel level access unfortunately. The Linux ones including crowdstrike and also the Open Source tools SELinux and AppArmor all need some kind of kernel module in order to work.
crowdstrike has caused issues like this with linux systems in the past, but sounds like they have now moved to eBPF user mode by default (I don’t know enough about low level linux to understand that though haha), and it now can’t crash the whole computer. source
As explained in that source eBPF code is still running in kernel space. The difference is it’s not turing complete and has protections in place to make sure it can’t do anything too nasty. That being said I am sure you could still break something like networking or critical services on the system by applying the wrong eBPF code. It’s on the authors of the software to make sure they thoroughly test and review their software prior to release if it’s designed to work with the kernel especially in enterprise environments. I am glad this is something they are doing though.
At least SELinux doesn’t crash on bad config file
I am not praising crowdstrike here. They fucked up big time. I am saying that the concept of security software needing kernel access isn’t that unheard of, and is unfortunately necessary for a reason. There is only so much a security thing can do without that kernel level access.
Does anything running on an x86 processor from the last decade?
Tell me you don’t understand what it is by telling me you don’t understand what Crowdstrike does.
I mean are we surprised by any of this?
I don’t remember much about my OS courses from 20 years back, but I do recall something about walls between user space and kernel space. The fact that an update from the Internet could enter kernel space is insane to me.
You mean you don’t update your kernel?
I do, but I do it on my terms when I know it is stable. I don’t allow anyone to push updates to my system.
Agreed. Point is, I’m pretty sure programs in kernel space can still read stuff in user space, which can be easily updated.