Published December 1, 2020

This article is part of the Flight Test series:

Good software is expensive, but bad software is far more expensive.

Bad software is expensive, but it is sometimes hard to spot the bleed in money. First of all, engineers are hard working and creative. They will use whatever means necessary to find a solution to process their data. They will use Python, MATLAB, or Excel. They will write code and develop macros and invent their own solutions to problems.

Unfortunately, the engineer sitting next to them will also be busy finding their own unique solution to the same problems. Ask your engineers where they spend the most time, and which of their applications are the hardest to use. Their response will provide insight into where your software improvements should focus. Most, if not all, software improvements in support of daily data reviews and report generation are actually free to an organization. We say this because well designed software will save an organization more in labor hours than they spend on the software development. The question is not ‘will it save us money?’, but ‘what is the payback period?’.

Good software improves the quality of your test organization as a whole. The easier it is to create and view plots, the more familiar the test engineer will be with their data and their aircraft. They will understand their systems. They will identify unreliable data. They will never miss a limit exceedance. They will have more time to plan the next flight.

Safety

The greatest benefit will be to safety. A test organization can only hire a limited number of engineers. Those engineers must write test plans, write flight cards, brief flights, monitor flights in telemetry or on-board, write their daily report, conduct data reviews, and write reports. The more effort is dedicated to daily data reviews, the less time is available for activities that contribute to safety. An engineer that is struggling to get the day’s data review completed won’t have time to double check how their control margins are trending for the next flight. Simply put, any and all organizational inefficiency directly reduces the operating safety of a flight test organization. For the FTE, data review efficiency is paramount to keeping their focus on tasks that will protect their flight crew and allow for successful flights.

Flight Hour Reduction

Another benefit will be seen in flight hour reduction. Test team A started out using Excel to process their rotorcraft engine performance data. By regulation, this data had to be collected during dynamic maneuvers with stabilization periods of less than 15 seconds. The pilot must fly a target airspeed and descent rate, then fail an engine, then pull to within 1% of a target engine power and stabilize the engine within 15 seconds. The best test pilots will be challenged to fly this maneuver correctly. To make matters worse, each test engine has only 200 seconds of engine life at this power. Too many mistakes will lead to premature test engine overhaul, weeks of delays, and hundreds of thousands of dollars in maintenance expenses. Using Excel as the data processing tool, the test team was unable to keep up with the data. They flew their data points, but could not analyze the data immediately to see if they were successful. Test errors were not seen until after the team returned from offsite testing. The engines were at end of life, but the data proved to be inadequate.

If your test team is making mistakes, it is either that you need better training, better people, or better tools. Of those three, new tools are the cheapest. The test team invested in better software. They began computing engine performance margins in telemetry. They saved on average 2 minutes of flight time per test point. Two minutes is important with hundreds of test points. They processed their test data every evening before finalizing flight cards for the next day. Any errant data points were repeated. They reviewed control inputs with their pilots daily and coached them on the correct inputs. They nailed their test points and came home with their report already written demonstrating compliance.

What did the software cost that achieved this feat? About 1/10th the cost of a single engine overhaul, or about the same amount of money that was spent on a single test flight, money that was saved by reducing the test time per data point by 2 minutes. Money well spent, and a sound investment for the future.

Better Telemetry Monitoring

Test team B was conducting extended test operations on a very complex aircraft. They were monitoring each test with a test director, avionics/electrical engineer, flight controls engineer, powerplant engineer, fuels and systems engineer, structural loads engineer, and a dynamics engineer. All stations were safety critical. The team was being asked to increase their test pace and went to 2 shift operations 5 days a week and expected to maintain that pace for a year. Unfortunately, they had insufficient engineers to staff both shifts. A better way was needed; a better telemetry software was needed. The fundamental solution was to ask what was hard, and then find a way to make it easy. Most importantly, total operating safety must increase, not decrease.

Monitoring tasks were automated one by one. Engineering displays were standardized and only allowed the necessary level of customization. Think of it like getting into an aircraft. Aircraft displays will have a familiar layout, common color coding, etc. The primary task of the display is to make available the needed information while reducing workload to the minimum. The developer asked the question, what does the engineer need to see or do here, and allowed them to make that happen with no more than 1 click. No menus and submenus; if it had to be done in flight it took at most 1 click, no more.

A standardized engine pod was created that had just enough flexibility, but was otherwise rigidly maintained a common layout. Most importantly, the engine displays alerted the test team before crossing maintenance limits, every time. Fuel monitoring was automated with the software predicting the time remaining in each test window and allowed the engineer to see which fuel tank selection offered the longest test time in the window. The software alerted the test team at the moment when they needed to return to the field to land at their desired minimum fuel state. The Test Director took over monitoring for engines, fuel, and mechanical systems.

Implementing ground-based logic on telemetry data is a challenge. Telemetry data is noisy. You can’t report an engine fire every time a couple bits get swapped in the data stream. To be successful, the software had to detect and ignore noisy or invalid state data. Avoiding false positives was critical. Not being annoying was critical, especially for the next effort. A Flight Test Warnings, Cautions, Advisories, and Maintenance screen was created which automated all system health monitoring for Flight Controls, Avionics, and Electrical systems. Emergency Procedures, Correction Actions, and Flight Continuation/Abort Criteria was loaded into the software, available with 1 click when the test system detected an anomaly. A Flight Test Aural Warning Generator was also added along with a hard deck “RECOVER ALTITUDE” alert and engine “CHECK POWER” alert to ensure attention was directed to critical changes in aircraft status. The recover altitude alert was predictive, monitoring both the test altitude and descent rate to trigger an alert at a defined number of seconds before the hard deck breach. The Test Director then took over for Flight Controls and Avionics/Electrical.

Refinements were continuously added to the application, and the Telemetry team was reduced from 7 engineers to a minimum staff of two. Each shift was staffed with three engineers despite only needing two so that there was extra capacity to respond to abnormal conditions. Was the goal of maintaining operational safety maintained? Without a doubt. Many crews received timely guidance made possible by system automation, but one crew in particular is thankful for the “RECOVER ALTITUDE” alert.