The FBI spent $170 million on broken software, which it has since scrapped. Now it's planning to spend $450 million on, one hopes, working software:
Because of an open-ended contract with few safeguards, [San Diego-based Science Applications International Corp.] reaped more than $100 million as the project became bigger and more complicated, even though its software never worked properly. The company continued to meet the bureau's requests, accepting payments despite clear signs that the FBI's approach to the project was badly flawed, according to people who were involved in the project or later reviewed it for the government.
David Kay, a former SAIC senior vice president who did not work on the program but closely watched its development, said the company knew the FBI's plans were going awry but did not insist on changes because the bureau continued to pay the bills as the work piled up.
Along the way, the FBI made a fateful choice: It wanted SAIC to build the new software system from scratch rather than modifying commercially available, off-the-shelf software. Later, the company would say the FBI made that decision independently; FBI officials countered that SAIC pushed them into it.
Upton Sinclair's wisdom notwithstanding, consultants have an obligation to inform clients about problems before they become too large to solve. Consultants also have an obligation to make appropriate build-or-buy recommendations to clients; in this case, if SAIC made such recommendations, there doesn't seem to be any evidence.
On the other hand, the Post article suggests the FBI had almost no clue what they were doing, bolstering SAIC's claims that they told them so.
Still, even assuming the best possible facts in SAIC's favor, they should have done the right thing, whatever that "right thing" was at any point in the relationship. Like, for example, testing the software, even if the FBI didn't think testing was important.
When a project like that blows up, everyone looks bad. Sometimes the consultant just has to walk away before that happens.