How secure is software X?
At least as secure as Vulnerability Assessment Assurance Level P; or Q or R.
Well, that's what I think we should be able to say. What we need is an open
standard, that has been agreed upon by recognized experts, against which the
absence of software security vulnerability can be measured - something which
improves upon the failings of the Common Criteria. Let's choose web server
software as an example. When looking for flaws in a new piece of web server
software there are a bunch of well known checks that one would throw at it
first. Try directory traversal attacks and the several variations. Try
overflowing the request method, the URI, the query string, the host header
field and so on. Try cross site scripting attacks in server error pages and
file not found messages. As I said, there's a bunch of checks and I've
mentioned but a few. If these were all written down and labelled with as a
"standard" then one could say that web server software X is at least as
secure as the standard - providing of course the server stands up.
For products that are based upon RFCs it would be trivial to write a simple
criteria that tests every aspect of the software as per the RFCs. This would
be called Vulnerability Assessment Assurance Level: Protocol. If a bit of
software was accredited at VAAL:Protocol then it would given a level of
assurance that it at least stood up to those attacks.
Not all products are RFC compliant however. Sticking with web servers, one
bit of software might have a bespoke request method of "FOOBAR". This opens
up a whole new attack surface that's not covered by the VAAL:Protocol
standard. There are two aspects to this. Anyone with a firewall capable of
blocking non-RFC compliant requests could configure it to do so - thus
closing off the attack surface - from the outside at least. As far as the
standards go however - you'd have to introduce criteria to cover that
specific functionality. And what about different application environments
running on top of the web server? And what about more complex products such
as database servers? I suppose at a minimum for DB software you could at
least have a standard that simply checks if the server falls to a long
username or password buffer overflow attempt and then fuzz SQL-92 language
elements. It certainly makes standardization much more difficult but I think
by no means impossible.
Clearly, what is _easy_ is writing and agreeing upon a VAAL:Protocol
standard for many different types of servers. You could then be assured that
any server that passes is at least as secure as VAAL:Protocol and for those
looking for more "comfort" then they can at least block non-RFC compliant
traffic.
Having had a chat with Steve Christey about this earlier today I know there
are other people thinking along the same lines and I bet there are more
projects out there being worked on that are attempting to achieve the same
thing. If anyone is currently working on this stuff or would like to get
involved in thrashing out some ideas then please mail me - I'd love to hear
from you.