Shutdown -P now in the wrong terminal window and a server at the other side of the country went down.
Electrical work and blacked out the entire 500m level in the mine.
There were three women who were best friends, took their breaks together, etc. And in the Christmas season they wore matching knit sweaters and would walk down the hall side by side so it would read “Ho! Ho! Ho!”
But one day when I was leaving the break room, they approached… and one was out sick. Before I could stop myself I asked “Where’s the other Ho?”
Might’ve gotten a visit to HR from it if I hadn’t looked so shocked at myself.
Honestly, they were inviting that one upon themselves.
Nice try, HR.
Worst thing in the office place was when some idiot left their window open in the middle of Winter, temps fell below 0F with high winds, and froze the 2" sprinkler pipes running over their office. Flooded most of the 2nd floor then started running through and raining out onto the 1st floor (and then into the basement). And it happened during covid lock-downs so it was fortunate anyone was even in the building to report it.
My own personal oopsie was checking network cabling in a small room, bent over to check things low and then wandered out to check elsewhere… Then noticed there was a LOT of commotion on the sales floor. Turns out I hit the power switch on one of the phone cabinets with my ass and shut down half the phone lines.
Did you own up to it or just turn it back on?
Oh I came clean. We were actually trying to figure out HOW it happened so we could try and prevent the same issue in the future.
“no thicc booties allowed in here” sign
“Attention, trailer swings out!”
Accidentally hitting reply-to-all on a company wide email and more or less stating that I wanted to be transferred to another team.
There was a new team forming elsewhere, and in fairness, it was a great opportunity in a lot of ways. But… I didn’t get the transfer until another batch of jobs opened a few months later.
That… was a long few months.
I didn’t actually delete the data but for a solid 1min I thought I had deleted an entire production db of data.
I made a delete then I hit refresh and nothing came. I refreshed again and no records panic started to set in and I refreshed again and still no records. I knew that changes replicated over to our quick backup every min so I picked up the phone and when the guy answered I said I need you to turn off the replication right now I think I just deleted the ministry of health.
After a bit of troubleshooting it turns out the data was fine and my delete worked as intended. The issue was in my client and we checked a few things then gave up. I went for a long lunch after that.
The biggest actual mistakes I’ve done were all caught by a really good manager i had and so I can’t even remember them because they never blew up.
Most recent, but not the absolute worst, was ripping my pants at work. I bent down to pick something up and heard the rip. It was over my crotch region too. Thankfully I had boxers on but was still pretty embarrassed.
Thankfully my boss was cool about it and I just drove over to Costco down the street and got a new pair and changed in the back of my car. He make a joke when I got back which was fine.
-
Alt tabbed once too many times, clicked drop database and yes. Deleted the live authentication DB for America’s Army: Operation video game.
-
Missed the word “add” in “switchport vlan add” on a switch, overwriting the list instead of appending to it. Took out the only connection between two datacenters we were in the process of migrating between. Took me 14 minutes to run to the datacenter, plug in a console cable and fix it.
Similar to your #2, but less serious, I once wrote a script to power down virtual machines for a data center move. It was a nice piece of work too, grouping them in batches, sending shutdown commands to the guest OS, falling back to forcing a power off through the hypervisor after a configurable timeout…
I don’t recall the specifics of the problem or the virtual infrastructure I was working with, but in short I didn’t have sanity checks on what was being shut down. Ended up force shutting off the hypervisor/virtual infrastructure management system.
Added an extra few hours the move with that.
I can absolutely see that happening in vsphere.
-
Many years ago, I worked in a call center. I was sitting with someone who was new helping them take calls and both of our headsets were plugged into the phone. The trainee was helping a store employee and she was just being awful to him. While she went to get something from the customer, I muted the line and said, “God, what a removed!” except my finger was hovering over the button and I hit it just in time for her to hear me say removed. I fully panicked and hung up on her. Nobody ever said anything to either of us and this was back when landlines would occasionally cross, so hopefully she thought that’s what happened since she hadn’t heard my voice up until then.
Anyway, I’m pretty sure that’s the origin of why I still don’t trust mute or hold to this day. I’m not talking shit until I know that call is disconnected.
The one I still feel guilt over was a time when i found out someone had left an animal trap loaded when they left for vacation. There was a live raccoon in it. I know I shouldn’t’ve carelessly opened it, but I should’ve done something. Even killing it would’ve been kinder. I carry that one with me, to remind me to act when I can. I’m still bad at it, but I try.
The other day I told a customer I could smell gas in her apartment, and even though I feel like a dumbass because it wasn’t a leak (probably lingering smell from them moving an appliance and hitting it on and off by accident), I don’t regret mentioning it. Sometimes I just am going to be an obnoxious jackass about that stuff.
For the 2nd one, better safe than sorry. You would feel like more of a dumbass if you heard a story in the news about a gas leak fire, or some other form of damage/injury/death.
I recently had a gas leak scare. Originally thought it was plumbing. They checked for gas leaks with a tool and found nothing. No obvious plumbing problems. We called the fire department the next time we smelled it to be safe. No gas leak but a bad car battery being charged was releasing a sulfur smell and about to catch fire.
Tripped and dropped a box, worth approximately $220,000 today, of extremely precise tooling meant for a cutting die. I was on my way to my bench to wrap them up safely. Boss was not pleased that day.
First I was like wow, then I started to count how many zero’s there came afterwards. Ouch that hurts.
Worked, after hours from home, on a Windows Server and fixed and issue with the Database on there. After doing so i thought I’d go to bed and shut down the machine… only I hadn’t yet left the RDP connection and shut the server down by Accident. Had to drive to work and start the server up again.
Accidentally shutting down a Windows server is impressive. They have that “why are you shutting down?” dialogue to prevent this scenario.
And there are ways of having that entry removed from the list of options entirely, and not just shifted to the drop-down menu. Makes it harder to physically shut down, but its absence can be a WTF big enough for you to realize which machine you are working on.
I don’t bother doing that to VMs, which can be trivially restarted, but their Hyper-V hosts? You betchya I do it to those.
Oh I was totally on autopilot and selected “Maintenance (Software)” because that is what I did… and I discovered the brainfart two seconds too late.
- Put down a bottle of bleach a little too quickly, a little spurt splashed exactly straight up out of it when it hit the floor and somehow hit me right in the eye. Washed it out in the sink and finished my shift with my eye bright red, instead of, IDK, going to the fucking doctor like I probably should have, because I was young and exploitable.
- Not me but a coworker: Found a handgun in our customer’s stuff, started messing around with it. You know the punchline. The bullet went through a few walls, cops got called, he made up some story about how it went off on its own while still in the cabinet which no one believed, somehow still didn’t get in legal trouble. He got fired over the phone before he had even left the customer’s premises.
- Got hired to a startup to fix the intranet slowness, started work as everyone was leaving, instantly fucked the router and broke the network completely for the whole floor, couldn’t fix it for hours and stayed there in a panic until about 3-4 in the morning when I finally figured it out, and fixed the network and the slowness both. Never told them anything except the ending, and they liked my work and hired me full time.
- Fucked the partition table to the main production server and my boss who was sitting right next to me had a mini panic attack while I reconstructed it from my notes and all the filesystems came back. Keep a notebook, it’ll save your ass.
Haha loved N.3
How old was the guy in N.2? Sounds like such a dumb kid thing to do
Maybe 20 years old. He was waving it around at the other guys on the job, too, joking around with it. He was lucky he didn’t kill someone and have to live with it for the rest of his life.
My dad told me when I was a kid: If you’re ever in that situation, just stand up and leave. Don’t say hey don’t do that. Don’t wait around and hope everything is okay. Don’t start joking around about it too. Just don’t say a word, stand up, walk out of the building, go somewhere else, the end.
Your dad gave you some solid advice.
Years ago, when projectors were common in conference rooms. Someone was giving a practice presentation before the real deal in front of 80+ audience members. It was just our team of 8 or so the room for the dry run. In the middle of the presentation, there was a terrifyingly loud POP sound as the bulb blew out in the projector. It scared the shit out of everyone in the room. We all laughed after the initial shock wore off.
One of my coworkers stepped up on a table to take a look at it. I was near him and I waited for silence in the room while he was fucking with the thing, and clapped my hands together very loudly, simulating the previous scare. He let out a shriek of terror and clutched his chest. Everyone laughed. Eventually he laughed as well, but said something like “damn, that scared me.” Within a week he had a legit heart attack.
He was ultimately okay, but I still think about it. I know I didn’t cause it, but for a long time I couldn’t shake the guilty feeling that I contributed to it. Oops. Sorry Ken.
Pushed to prod
