Common mistakes space operators make and how to prevent them

WHAT YOU’LL READ IN THIS ARTICLE

• The mistakes that keep space operators up at night

• The truth about space operations in the 21st century

• Solutions for common space operations mistakes

• How procedures need to evolve to adapt and scale for increasing demand

• Common fundamental mistakes complex operations engineering teams make

• How to prevent fundamental mistakes and improve risk management

As a complex operations team, what keeps you up at night? When it comes to risk management, we’ve all heard about, and maybe even experienced, the worst-case scenarios. The smallest mistake can have profound costs, both human and financial. Many of us have heard the horror stories.

3 Historic Costly Space Operator Mistakes

• A misplaced hyphen in the code caused the guidance system to falter on the Atlas rocket launched with NASA’s Mariner 1 Spacecraft in 1962, resulting in a destructive abort and costing the equivalent of $720 million.

• The Soviet Phobos 1 met its doom in 1998 when test-mode software missed a single character and was not removed after testing on earth. Mission operators lost contact with the spacecraft, and it lost its orientation towards the Sun. Battery power ran out, and that was the end of Phobos.

• The Mars Climate Orbiter disintegrated in 1998 when two mission engineering teams were using different measuring units for their calculations–one team was using metric, and the other team was using imperial units. 

THE TRUTH ABOUT SPACE OPERATIONS IN THE 21ST CENTURY

Most people would assume that with all the space hardware innovations we’ve had over the past few decades, these kinds of human error accidents wouldn’t happen anymore. And yet many of us know that space operations disasters are better prevented at the testing, integration, and operations stage–long before launch ever happens.

As much as space hardware has evolved and improved, most complex operations engineering teams are still relying on outdated procedures that haven’t improved risk management. Oftentimes teams are still relying on a haphazard and scattered documents in Word, PDF, and spreadsheet platforms. More shocking are the teams who are still utilizing paper procedures for their business. 

SCALING SPACE OPERATIONS AND PROCEDURES TO MEET DEMAND

The good news is there are simple, elegant, and cost-effective solutions for teams who want to scale and meet the rising demand of the future for free-flying space vehicles. 

SpaceNews reports that roughly 17,000 satellites will launch to non-geostationary orbit between 2021 and 2030, compared with around 3,800 spacecraft across all orbits over the previous decade. 

Relying on outdated procedures for testing and integration will increase risk of mistakes and mission failures. Converting to digital tools that integrate with multiple command and telemetry platforms, data sources, and post-processing tools allows operators to automate decisions based on data and provides the best investment for teams who want to scale.

SOLUTIONS FOR COMMON SPACE OPERATION MISTAKES

1. Train Like You Fly

Ensure your team is well-trained for both nominal and off-nominal scenarios. This might seem too obvious, but we've seen teams with no training fail to execute on an operations plan. 

Typically requirement specifications only refer to how the system should behave under normal circumstances. Is your team adequately trained and equipped to specify how the system should behave if it cannot or doesn’t behave as normally expected?

We often see engineers underestimate the probability of rare occurrences and are caught off guard in these rare circumstances. This can be avoided if these off-nominal conditions are identified and documented early in the life-cycle of the mission.

Make sure your team answers these questions early on:

• How will the system recognize off-nominal combinations of conditions and events?

• Does the system need sensors to determine the existence of these states or occurrence of these events?

• How available, reliable, accurate, and precise must these sensors and inputs be?

Additionally, be cautious when coding and avoid using certain programming language features or copying code from the Internet that may cause your software to be unreliable, unsafe, or insecure.

2. Operate Early and Often

Spend time thinking about operations early and often in the program life-cycle and talk about how things will work before you dive into Integration & Test and eventual operations. 

Creating and communicating clear unit and integration testing in an agile environment can help your team identify problems early on and also provide you with time and space for creative solutions that drastically increase the likelihood of a successful and elegant mission. 

3. Data, Data, Data

Gather, capture, record, review, repeat. You should always have a record of what you do, even if it's in training or a simulation. Without records of that data, your team won't grow and learn from their mistakes. 

Be open and honest about mistakes. Mistakes are ok, but you can't make the same mistake twice, especially as demand for well-planned missions increases over the next few years. 

It’s incredibly valuable to have a track record of every adjustment that was made so that your team can refer back to incidents when they need to recall what the problem and its solution were. The data is there to collect every piece of information so our already-crowded human brains don’t have to struggle to remember on our own.

4. Invest in Infrastructure

We often don’t see the seemingly insignificant mistakes until the big disasters happen. Prevention of mistakes in the future on every scale begins with addressing the fundamental mistakes and assumptions teams make about their procedures and available digital tools.

For instance, the misplaced hyphen that sent NASA’s Mariner 1 to its doom wasn’t the fundamental mistake. This human error could have been avoided if the team had access to digital tools and software for complex operations that would have drastically improved their testing and integration procedures.

Here are the fundamental mistakes we see space operators make: 

4.1 RELYING ON OUTDATED AND INEFFECTIVE DIGITAL TOOLS FOR COMPLEX ENGINEERING AND OPERATIONS

Change can be hard–especially for small teams with limited budgets. If your team has been able to avoid major and costly disasters up until now using Word, Excel, Confluence, and OneNote, count yourselves lucky. 

As industry demand increases, your team will need to adapt in order to scale and improve risk management while also eliminating human error. Trying to organize multiple platforms manually will inevitably lead to costly disasters as project volume increases.

We get it. Teams are using these habitual platforms because they’re inexpensive and team members are familiar with using them. However, we know that, if given the option, most teams would welcome a more efficient digital toolkit that would give them not only the ability to scale and improve risk management but would give every person on the team something priceless: peace of mind. 

4.2 USING PAPER PROCESSES TO DOCUMENT OPERATIONS, TESTING, AND PROCEDURES

Who’s still using paper procedures? It's a common issue, and there are more complex operations teams than you think still completing tasks on paper–which leads to more risk for human errors and accidents. That’s why simply converting to a digital process will give you an incredible lead and competitive advantage when it comes to winning contracts for your business as the space industry expands exponentially.

If you’re on a team that’s still utilizing paper, you’ll be surprised at how quickly you can adapt to using cloud-based digital tools to manage risk, reduce manpower/hours, and strive to eliminate human error.

When you invest in complex operations software for your engineering team, all those extra hours gained can be used to innovate, take on more contracts, and scale your business in a sustainable way while minimizing mistakes and avoiding nightmarish disasters.

4.3 WASTING RESOURCES ON DEVELOPING COMPLEX IN-HOUSE OPERATIONAL SOFTWARE

Many large space operation firms recognize the organizational nightmare that becomes an engineering team's biggest hurdle as they bounce around from Word to Excel to OneNote.

It’s incredibly tempting for companies to throw a large budget toward creating a customized digital toolkit to streamline their procedures and minimize risk.

However, these efforts almost always continue indefinitely and require much more of, well, everything. If your team decides to develop in-house software for your company, here are some of the common issues we see teams run into:

•  The cost is never static. The scope of the project will increase, and the cost will increase as new features are added.

• All the time engineers and operators must dedicate toward testing and feedback as the in-house software is developed takes away from time spent on mission operations.

• Your unique in-house software makes collaborations with other teams from different firms difficult and costs time and energy, increasing the likelihood that something could go wrong.

We’ve learned from experience that in-house tools are endless projects with innumerable uncertainties that can and do occur. Your team needs to focus on mission operations now and in the future, not tool development. 

With the increase in demand for space operations, it’s difficult to justify allocating the financial and bandwidth resources towards complex operations software–especially when a product already exists to get your team working more efficiently.

At the Heart of Epsilon3 and Why We’re Passionate About Contributing to Mission Success

By now it’s probably clear to you that there’s a better way to improve risk management while also scaling your team’s capacity to meet growing demands. And you guessed it, it’s the entire reason we created Epsilon3–software to manage complex engineering, testing, and operational procedures.

The first and most crucial step to preventing operational mistakes is to shift as soon as possible from outdated procedures and adapt to software that will help you in all the ways below:

Save Time and Money with Integrative Software

With Epsilon3 you can create, revise, and track procedures and tasks with critical mission data collaboratively, all in one place and in a standardized way.

Reduce Errors for All Complex Missions

Our software ensures no critical step or data gets missed and prevents failures with intelligent error checking and automation. 

Streamline Mission Communications

Epsilon3 guarantees that everyone on your team is always on the same page and knows who is doing what, when, and where. Responsibility is clear and well communicated.

Continuous Improvement for Efficiency and Performance

When you use Epsilon3, you can create detailed metrics and reports that boost operational efficiency and performance.

Upgrade to Better Complex Operations Software

In short, preventing costly mistakes not only saves your team time, money, and reputation, but will help you scale your operations in a competitive market with simple and straightforward solutions to meet your company’s needs and goals.

Mistakes like the Phobos 1 and Mars Climate Orbiter simply shouldn’t happen anymore. Peace of mind leads to more innovation and more successful space operations that are highly unlikely to fail.

Here’s to less risk, minimized wasted hours, and the elimination of human errors!

Want to schedule a demo of Epsilon3 and learn how it can help your team streamline testing and operations procedures? Click here to make an appointment with one of our team members.