How many people have a bunch of PCs that are down right now?

3 Likes

Not me. :man_shrugging:

12 Likes

We use CS… checked my windows laptop and it’s fine, but who knows.

Glad I’m out today :grin:

3 Likes

Same here. I hear a lot of the debit card machines are down.

1 Like

One of our IT guys installed CrowdStrike on our Ignition production gateway a few weeks ago for testing, and now we have the BSoD.

5 Likes

It's scary how easy it was to take down so many systems from such a simple attack, I mean mistake.

4 Likes

Perhaps your IT guys need to read some of this:

https://forum.inductiveautomation.com/search?q=%40pturmel%20malpractice%20order%3Alatest

and reconsider their OS choice(s). At least on production systems.

6 Likes

Reportedly, after the second failed restart, you'll get the option for advanced troubleshooting options. If somehow, you can get the boot to safe mode option to work, then you can delete file C-00000291.sys, and that will fix the computer.

2 Likes

Perhaps this should be under General Discussion?

Agreed. I moved it.

1 Like

I was converting a VHD to VMDK overnight and I walked into to my coworkers asking if I heard about the crowdstrike news and then a blue screen lol. Luckily got in after getting my bitlocker key and deleting the problematic file.

Most of our computers were off overnight so no problem but not me (:

My heart goes out to any and all IT working OT this weekend.

Who's idea was it to push on a Fridays anyways?

Lot of interesting stories - Reddit - Dive into anything

2 Likes

Standard Operating Procedure

1 Like

One comment from the thread I linked -

The entire sum of everything that Crowdstrike might ever have prevented is probably less than the damage they just caused.

I think they can remove the word probably though. I was at the doctors office this morning and they were down. Airports grounding flights, hospital servers are down. If no one knew this was crowdstrike they would think this was the single most effective cyber security attack ever.

5 Likes

Pour one out for the employee who pushed this update and is sweating absolute buckets right now :rofl:

17 Likes

Just curious @pturmel if you use any AV/security software on Linux or if you've ever felt the need to?

1 Like

We restored our gateway, but now my Kepware license is broken for some reason, ugh.

2 Likes

Haven't felt the need. Played with tripwire some, but there simply isn't strong evidence that Linux AV tools protect Linux beyond what a iptables and SSH certificates security provide.

There are some that are valuable for protecting Windows systems that use Linux as a fileserver or similar resource, and to add layers to susceptible web server technologies (looking at you, PHP). Out of date webservers are responsible for ~99.9% of all Linux compromises in the real world, for which Linux AV appears to be useless.

Probably the most valuable tool for publicly exposed servers is fail2ban, which leverages application logging and iptables to chop off IP addresses that probe your system.

I should add that SELinux is good for exposed systems, but it seems to only be a speed bump for any hacker that gets root access.

7 Likes

I've one rule. Don't use the last version always use penultimate one :wink: That rule saved me already a lot of time with some Rockwell software :smile:

1 Like

The crowdstrike update was automatic. You didn't choose to update or not it just downloaded and ran the update. Maybe there's a way to disable that, or if its offline it wouldn't, but the default installation does this, and I would bet they sell it as a better way to do things so you don't have to worry about staying up to date on security threats. It just works better! Until...

2 Likes

I just got back from a plant. We didn't put CS on their infrastructure but someone else did.

Main SCADA was down.

Fixed now.

2 Likes