How Incident Responders Can Manage Mental Health in High-Stress Tech Environments

SRE at Mews, apart from incidents and observability also really into fantasy books

Incident responders are key to keeping software systems secure and functional. However, the nature of this work can be extremely stressful and demanding, so it’s essential to manage your mental health while handling the challenges of incident response.

Here are some strategies I use myself to stay in tech instead of opening a florist shop, which might be on the horizon given my great track record in keeping plants alive.

Understand the nature of your work

Recognizing the high-pressure environment of incident response is the first step in managing your mental health. Incidents can arise unexpectedly, requiring immediate attention and problem-solving skills. Itʼs important to admit that this can be quite a significant stressor in your life.

There is an undeniable similarity with other types of first responders, such as firefighters, doctors, and rescue workers. Although the level of responsibility is incomparable, not many people think about this part of the job when choosing a career in IT. Understanding this can help you prepare mentally for the challenges ahead, and we can borrow some of the techniques from other types of first responders.

The feelings of grief and guilt that may arise when your application experiences a significant outage, along with the relentless thoughts about not identifying the issue sooner or not knowing how to resolve it right away, are not unlike what a doctor might feel in critical situations. While we are not literally saving lives (unless you’re fixing a bug in an airborne plane), we are certainly playing a role in safeguarding businesses and enhancing people’s daily experiences, which is meaningful in its own right.

If you’re a seasoned veteran, it can be easy to overlook how stressful the situation actually is; for newbies, but also for yourself. Do you remember how you felt during your first few incidents? Was it overwhelming and scary?

Some preach a “rite of passage” because they had to endure. If that’s your case, can you rethink this to create a more supportive environment? People have different levels of sensitivity, what they consider stressful, and how they respond to it. Without adequate support, stress can be the main reason people look for other opportunities.

Know how to support yourself

For many people, it may actually be easier to prioritize the protection of their team, but it’s equally important to safeguard your own well-being. Even in a highly supportive environment, the pressures of a stressful situation can take a toll on you, both physically and mentally. Recognizing these feelings before they escalate into burnout is crucial.

Before an incident

  • Get some practice beforehand in a safe environment: Participate in incident drills or join other teams’ incidents as a spectator.
  • Accept that incidents are inevitable: Of course, we try to prevent them, but it’s impossible to completely eliminate them. It can help to treat smaller issues as incidents to get practice. That way, it’s not so scary when a big issue inevitably emerges.

Establish a support system
At work:

  • Maintain good relationships with colleagues. A good team culture where people are willing to help and learn from each other doesn’t appear by itself.
  • When people are working remotely, it can be easy to disconnect from the actual people behind the screens. When possible, try to use video calls. It seems like a small thing, but you might be surprised how much closer that brings the team together. Also, investing in the team to actually meet in person and have fun together pays off.
  • Have good practices in place:
    • Guides and runbooks for typical situations: It can seem like paperwork to write down a manual for a simple service restart, but imagine the future-you, woken up at night. Is it really that trivial? Would you remember all the manual steps? What can be automated?
    • Observability: Do you have all the tools you need to investigate?
    • Access: Do you have sufficient access to observability tools or mitigation steps? Have you tested it?

At home:

  • This can involve having a supportive partner who ensures you eat something and get some rest. If you live alone, this can mean stocking up on snacks and maintaining a good daily routine.

Professional help:

  • Donʼt hesitate to reach out to mental health professionals if you feel overwhelmed. Nowadays, taking care of your mental health is as normal as going to the dentist. If you’re under extra stress at work, a therapist can guide you through the emotions constructively.

Self-care:

  • Prepare yourself for a stressful situation by feeling good in the first place with regular exercise, healthy eating, and adequate sleep. Donʼt underestimate these.
    • Stress-management techniques like mindfulness, meditation, and breathing exercises. Practicing mindfulness can help you stay grounded during high-pressure situations. Meditation doesn’t need to be spiritual; it can simply calm your racing thoughts and ground you.

During an incident

Taking breaks
TL;DR: You can totally take a break and ask for help. For whatever reason.

When an incident unfolds rapidly, it can feel like taking even a brief bathroom break or grabbing a glass of water is out of the question. If you find yourself in this situation, it’s crucial to recognize that many of these pressures are self-imposed. The sense of belonging and responsibility is great, but would you hold a colleague to the same unrealistic standard you impose on yourself?

Even typing this out feels outrageously ridiculous. Nobody in their right mind would tell you that you can’t go. It’s absurd. Just type “brb” in the chat and don’t overthink it. If you are the person leading the incident, you can also hand over your role (formally or informally, “Hey, Iʼll be back in five, the bridge is yours, Dave” → “What did I miss?”).

  • Take regular short breaks (bathroom, water & food, fresh air, recharge, go punch a pillow). If you need a longer break, tell your team and do an official handover.
  • There can be a lot written about sustainable on-call rotations and good practices, but not every organization/team has something in place for situations that haven’t happened yet. This can be super hard when you don’t have anyone you can officially hand over to, e.g., if you’re covering the whole weekend for passive on-call and everybody else is somewhere without internet access.
  • There might be legal obligations based on your country, e.g., if you spend Friday night troubleshooting an incident, you deserve a sufficient amount of uninterrupted rest. If this happens so rarely that you don’t have an official backup in place, remember you can always escalate up.
  • A family emergency should take priority. You also deserve enough rest if you were working at night.

Responsibility

  • You donʼt need to know and fix everything. You can involve more people who might have different perspectives, and you can figure it out together.
  • Don’t be afraid to improvise. Even partial solutions can be good enough to mitigate the problem.
  • When dealing with multiple issues at the same time, it can be overwhelming. Divide and conquer. You can delegate tasks, and not all tasks are equally important.
  • Avoid taking side quests. Sometimes the discussion takes a detour; it can be useful to bring the focus back.
  • If you are writing code to fix something and the incident call discussions are distracting, it can be helpful to drop from the call and report progress regularly.
  • Avoid blame culture. If you tend to look for someone to blame, try to reframe your mindset to look for opportunities to improve. Even when someone makes a mistake, it often points to a lack of control mechanisms. It’s never just one mistake, but a chain of events that can be analyzed afterwards. The incident call isn’t the right time or place to do a full analysis.

Ready to support us during incidents while prioritizing your mental health?

Check out our open positions!

After an incident

Self-care

  • Adequate sleep, especially if you were woken up at night. Get enough rest to compensate for the sleep debt.
  • Do something nice for yourself. It can be a massage, yoga class, or any kind of treat.
  • Participate in the postmortem or root cause analysis of the incident. You have the most context and knowledge, and your input can help prevent the same problem from happening again.

You can delegate some of your other responsibilities. Tell your manager or team clearly and specifically what you need. Typically, they are eager to help but don’t know how, or don’t want to overstep. What would make you feel less tired and burned out after a stressful incident?

  • Some time off?
  • Someone to talk to and vent?
  • Can someone take the ticket you are working on?
  • Can someone take your next on-call shift?
  • Can someone else drive a meeting that you normally drive?
  • Can some meetings be rescheduled?

Know how to support others

While you might not be required to drive incidents in your role, there are many things you can do to support your colleagues and help them reduce the stress that these situations bring. Try to be observant and offer help where you can, while being mindful of your own boundaries.

How to support a colleague as a peer
It’s hard to ask for help, especially when you don’t want to burden others during a crisis. You can actually do a lot for a peer during and after an incident to help them:

  • Check on their well-being: Sometimes you might not be able to help technically, but that doesn’t mean there aren’t other ways to support.
  • Food and water: Our field includes many neurodivergent people, and it’s easy to forget basic needs.
    • Get them a glass of water or a small snack from the kitchenette.
    • If the incident is running over lunchtime without a resolution in sight, offer to order some food. Some people may decline due to stress responses, but the gesture matters.
  • Normalize breaks: Even when remote, you can normalize taking breaks by sharing that you’re taking one.
  • Offer help: Try to think of something specific you can take off their plate. “I will take care of X” puts much less of a mental load on them than “tell me what to do.” This applies to both incident tasks and regular responsibilities.
  • Try helping with the incident itself:
    • When some action is needed and you know how to do it (updating status, restarting a service, etc.).
    • When you don’t know how, pair with the person doing it so you can learn for next time.
    • Join the brainstorming, you might have a fresh perspective.

How to support a colleague as a manager/leader
Iʼm not a manager, so I’ll share from the receiving end (or what I wish my managers had done). This includes all of the same things you would do for a peer, plus:

  • Help manage priorities so no one is overwhelmed: As a manager, you have a clearer view of the team’s workload. You can identify which tasks are most urgent and may need to be reassigned.
  • Encourage breaks: If the incident stretches outside of working hours, encourage team members to take breaks or time off in the following days to prevent burnout. Self-care during such times is crucial.
  • Bring a “How can I help?” mentality: You likely have the power to escalate issues to other teams or third parties, which can make a significant impact. Plus, your technical expertise might be exactly what’s needed.
  • Avoid putting blame on anyone: Remember, those working on the incident are actively trying to solve the problem. Your presence should be one of reassurance, not added stress.
  • Appreciate team efforts: Acknowledge the hard work and dedication they are putting in during a crisis.
  • Sometimes, preventing a crisis is just as important as resolving one: Recognize when someone’s proactive efforts are preventing bigger issues and allow them the time and space to keep doing that vital work. It’s not always glamorous, but it’s necessary for maintaining stability.

ℹ️ Would you like your colleagues/leaders to offer you help when youʼre getting overwhelmed during an incident or drained after one? Did you try asking them specifically for what you need? Typically, they are eager to help but donʼt know how or donʼt want to overstep, or you might just look like an absolute badass who has everything in control.

Summary

Staying calm during incidents is a valuable skill, not just for tech crises, but for handling any kind of emergency, from medical issues to accidents. It’s worth pushing through the discomfort to build it, and there are techniques that can make preparation and recovery easier.

I hope you find some of these strategies helpful and start using them in your own practice (get those runbooks written!). What helps you stay grounded during incidents and bounce back afterwards?

Curious about why we conduct incident drills—and why you should too? Check out Fina’s previous blog post to learn more.

SRE at Mews, apart from incidents and observability also really into fantasy books
Share:

More About