An improbable pitfall discovered in the Minotaur rocket's flight software, posing only a minuscule chance of ruining a mission, nevertheless will be patched before the booster launches a unique space surveillance craft, officials explained Thursday.

Credit: Orbital

Liftoff from the southern pad at Vandenberg Air Force Base in California had been scheduled for Thursday night. But the Space Based Space Surveillance (SBSS) satellite launch was placed on hold after rocket-maker Orbital Sciences found a timing flaw in the Minotaur's software while preparing a different vehicle for a planned September flight from Alaska.

"The anomaly occurred during preliminary testing for our subsequent launch of the Space Test Program's S26 mission. Our investigation pointed to a problem with the common launch vehicle software that's used on all our missions. This required that we assess impacts to the SBSS launch," said Col. Mike Moran, commander of the Space Development and Test Wing at Kirtland Air Force Base, New Mexico.

"To date, the government and industry team has performed troubleshooting, developed the needed software fixes and is now beginning formal testing of the software updates. When we are confident that the launch vehicle issue is fully resolved as demonstrated through successful testing, we will re-establish a launch date and resume normal launch processing."

The rocket stands stacked atop Space Launch Complex 8 awaiting shipment of the SBSS satellite to the seaside pad for attachment. The Minotaur 4 is comprised of Peacekeeper missile motors as its lower three stages and a commercial fourth stage for delivering satellites into orbit.

Concerns were raised when software testing revealed a potential launch failure scenario during simulations being run at Orbital's facilities in Chandler, Arizona. After ruling out other causes, engineers determined that the software itself had a bug.

"Initially, a test equipment issue or problem unique to the characteristics of the S26 mission was suspected. When no root cause was identified in these areas, it became an issue for the SBSS mission in mid-June. Since that time, the investigation has uncovered a timing issue in the flight software," said Tim Kettner, Orbital's chief engineer for the Minotaur program.

The issue was so tiny, however, that the glitch had never materialized in thousands of prior test cases.

"Orbital routinely subjects software to a rigorous testing regimen destined to expose statistically remote failure modes. This failure occurred in one test out of 1,000. No similar occurrence was noted for SBSS in over 4,000 runs," Kettner said.

"The test failure that we observed on STP S26 was a failure of the mission sequencing and a failure to successfully execute the mission in one of the 1,000 tests," Kettner said. "In the test that we saw, it did result in the halting of communications from the flight computer and a failure of that particular mission (simulation)."

Kettner said the timing error was spotted in the test scenario just after third stage burnout but could have occurred at any point during the launch sequence.

The odds of the problem actually striking a Minotaur during ascent is considered improbable but not impossible. So engineers have developed a corrective fix to the software.

"The fact that we did find this problem on the ground does not mean it would have occurred in flight. But it is a risk that our mission assurance process purposely intends to ferret out. So we are pleased that the process did that and that we have an opportunity to fix that problem, not just for the SBSS launch, but for the entirety of the Minotaur family," said Moran.

"We do intentionally subject our software suites to the rigorous and statistically significant testing for the expressed purpose of ferreting out statistically unlikely and remote failure modes. We believe that's exactly what occurred in this case. We're thankful it was discovered. Although we echo the sentiment it is extremely unlikely that it would have occurred in flight, we're grateful for the opportunity to fix it," Kettner added.

The flaw is located in low-level software that is common across the various rocket configurations in the Minotaur fleet. The updated software code will be employed to improve all other launches now under development.

"We have a released a software revision that addresses this issue and we are currently in the process of executing the formal testing of the revised software. Upon successful completion of this testing, we anticipate the software will be flightworthy and ready to support the SBSS mission," said Kettner.

It will take three weeks to perform the software re-testing before the final two weeks' worth of launch preps can be accomplished.

"We have laid out a rigorous plan there to execute 2,000 additional runs over and above what we would typically do for qualification of flight software. In addition to what we call the real-time closed-loop test, which is what we execute for qualification of software, we'll be running some stress testing to verify that timing conditions that we believe are at the root of this are adequately stressed and demonstrated robust for flight," said Kettner.

"Once we test the software in the factory and are satisfied it's fully flightworthy, we'll load that on the launch vehicle and then we'll run full system testing on the launch vehicle as well, as an added confidence and as part of our normal launch campaign," said Moran.

"The actual procedure to load and verify the new software takes between half-a-day to a full day. We can accomplish that and the rest flow, I think, within 14 days," said Kettner.

The SBSS satellite, currently tucked away inside a Vandenberg hangar, will be taken to the launch pad and mounted atop the Minotaur only after the software is deemed ready.

"Obviously we were hoping to be launching this evening, but it's always good to find issues on the ground," said Col. J.R. Jordan, SBSS mission director.

"It's always safer to be on the ground and fly tomorrow than risk something today. The fact that we caught this and we're correcting it is paramount to why we do business the way we do it."

A specific new launch date hasn't been selected yet.

"Our focus is on mission success," Moran said. "When we are ready to launch, we will."

Orbital's Minotaur line of rockets has been used in satellite launches and suborbital missile tests since 2000, performing all 17 of its flights to date successfully.