October 15, 2015
by Bruce Schneier
CTO, Resilient Systems, Inc.
Volkswagen and Cheating Software
For the past six years, Volkswagen has been cheating on the emissions
testing for its diesel cars. The cars’ computers were able to detect
when they were being tested, and temporarily alter how their engines
worked so they looked much cleaner than they actually were. When they
weren’t being tested, they belched out 40 times the pollutants. Their
CEO has resigned, and the company will face an expensive recall,
enormous fines and worse.
Cheating on regulatory testing has a long history in corporate America.
It happens regularly in automobile emissions control and elsewhere.
What’s important in the VW case is that the cheating was preprogrammed
into the algorithm that controlled cars’ emissions.
Computers allow people to cheat in ways that are new. Because the
cheating is encapsulated in software, the malicious actions can happen
at a far remove from the testing itself. Because the software is “smart”
in ways that normal objects are not, the cheating can be subtler and
harder to detect.
We’ve already had examples of smartphone manufacturers cheating on
processor benchmark testing: detecting when they’re being tested and
artificially increasing their performance. We’re going to see this in
The Internet of Things is coming. Many industries are moving to add
computers to their devices, and that will bring with it new
opportunities for manufacturers to cheat. Light bulbs could fool
regulators into appearing more energy efficient than they are.
Temperature sensors could fool buyers into believing that food has been
stored at safer temperatures than it has been. Voting machines could
appear to work perfectly — except during the first Tuesday of November,
when they undetectably switch a few percent of votes from one party’s
candidates to another’s.
My worry is that some corporate executives won’t interpret the VW story
as a cautionary tale involving just punishments for a bad mistake but
will see it instead as a demonstration that you can get away with
something like that for six years.
And they’ll cheat smarter. For all of VW’s brazenness, its cheating was
obvious once people knew to look for it. Far cleverer would be to make
the cheating look like an accident. Overall software quality is so bad
that products ship with thousands of programming mistakes.
Most of them don’t affect normal operations, which is why your software
generally works just fine. Some of them do, which is why your software
occasionally fails, and needs constant updates. By making cheating
software appear to be a programming mistake, the cheating looks like an
accident. And, unfortunately, this type of deniable cheating is easier
than people think.
Computer-security experts believe that intelligence agencies have been
doing this sort of thing for years, both with the consent of the
software developers and surreptitiously.
This problem won’t be solved through computer security as we normally
think of it. Conventional computer security is designed to prevent
outside hackers from breaking into your computers and networks. The car
analogue would be security software that prevented an owner from
tweaking his own engine to run faster but in the process emit more
pollutants. What we need to contend with is a very different threat:
malfeasance programmed in at the design stage.
We already know how to protect ourselves against corporate misbehavior.
Ronald Reagan once said “trust, but verify” when speaking about the
Soviet Union cheating on nuclear treaties. We need to be able to verify
the software that controls our lives.
Software verification has two parts: transparency and oversight.
Transparency means making the source code available for analysis. The
need for this is obvious; it’s much easier to hide cheating software if
a manufacturer can hide the code.
But transparency doesn’t magically reduce cheating or improve software
quality, as anyone who uses open-source software knows. It’s only the
first step. The code must be analyzed. And because software is so
complicated, that analysis can’t be limited to a once-every-few-years
government test. We need private analysis as well.
It was researchers at private labs in the United States and Germany that
eventually outed Volkswagen. So transparency can’t just mean making the
code available to government regulators and their representatives; it
needs to mean making the code available to everyone.
Both transparency and oversight are being threatened in the software
world. Companies routinely fight making their code public and attempt to
muzzle security researchers who find problems, citing the proprietary
nature of the software. It’s a fair complaint, but the public interests
of accuracy and safety need to trump business interests.
Proprietary software is increasingly being used in critical
applications: voting machines, medical devices, breathalyzers, electric
power distribution, systems that decide whether or not someone can board
an airplane. We’re ceding more control of our lives to software and
algorithms. Transparency is the only way verify that they’re not
There’s no shortage of corporate executives willing to lie and cheat
their way to profits. We saw another example of this last week: Stewart
Parnell, the former CEO of the now-defunct Peanut Corporation of
America, was sentenced to 28 years in prison for knowingly shipping out
salmonella-tainted products. That may seem excessive, but nine people
died and many more fell ill as a result of his cheating.
Software will only make malfeasance like this easier to commit and
harder to prove. Fewer people need to know about the conspiracy. It can
be done in advance, nowhere near the testing time or site. And, if the
software remains undetected for long enough, it could easily be the case
that no one in the company remembers that it’s there.
We need better verification of the software that controls our lives, and
that means more — and more public — transparency