From: "Geo." 

from Russ:

Howdy,

As you may have heard, I did a presentation last week at the 2004 AusCERT
Conference in Gold Coast, Australia.

My presentation was the culmination of analysis I performed on all
Microsoft Security Bulletins published by Microsoft from January 1, 2000 to
date. I analyzed the *vulnerabilities*, dissecting each bulletin into their
respective vulnerabilities. As we all know, each bulletin MS produces may
involve numerous vulnerabilities. In addition, vulnerabilities addressed by
a bulletin may affect some versions but not others. I tabulated all of this
information based on the facts in the bulletins.

The purpose of this was to address a common problem I see in the media,
namely, attempting to use the count of bulletins in comparisons with other
OS'. Patch count comparisons serve no purpose, but counting patched
vulnerabilities, IMO, is a more accurate comparison. In addition, I grouped
vulnerability counts according to whether they'd likely affect desktop,
server, and IIS servers. Again, comparing raw numbers for
"Windows" against other OS' isn't a correct comparison either, so
using numbers based on a role made more sense to me.

My presentation made no attempt to compare Windows to any other OS. I
compared MS to MS, NT 4.0 Workstation versus W2K Professional versus
Windows XP, NT 4.0 Server versus W2K Server versus W2K3. I compared IE 5.0x
versus IE 5.5 versus IE 6.0. For all, I made no distinction wrt Service
Packs (and therefore if security vulnerabilities were addressed by Service
Packs alone, they were not considered.)

Again, I'd like to stress, my analysis did not consider many things which
are important in determining the security of a system. For example, I did
not address severity, either as designated by MS or by me nor did I address
exploitability. Although my numbers did not include vulnerabilities which
were not installed by default, they did include vulnerabilities which could
have been avoided by configuration or by removing components. I did exclude
all vulnerabilities for things like SQL (including MSDE), Exchange, IIS
add-ons (like Media Server) and other server products.

I analyzed 452 vulnerabilities in 298 Microsoft Security Bulletins. I found
very little difference in the number of vulnerabilities affecting different
versions of a given product. The vast majority of vulnerabilities affected
all of the versions of a product I looked at, meaning to me that the
vulnerabilities were in legacy code carried forward into new versions.
That's the sort of stuff the Security Push was supposed to find, IMO, yet
there was nothing in my analysis to show the Push had an impact on this
problem.

In the period I covered, here are some of the stats I discussed;

1. Comparing Desktop OS' without browsers

NT 4.0 Workstation = 68 vulnerabilities Windows 2000 Professional = 86
vulnerabilities Windows XP = 55 vulnerabilities
NT 4.0 Server = 76 vulnerabilities
Windows 2000 Server = 98 vulnerabilities Windows Server 2003 = 24 vulnerabilities

2. Comparing the first 300 days of Windows 2000 Server to Windows Server 2003

Windows 2000 Server = 27 vulnerabilities Windows Server 2003 = 24 vulnerabilities

3. Comparing Windows NT 4.0 Server to Windows Server 2003 during the first
300 days of Windows Server 2003

Windows NT 4.0 Server = 22 vulnerabilities Windows Server 2003 = 24 vulnerabilities

This is where I made some comments about NT and Microsoft shareholders.
What I said was this. Some people think they should wait for an SP1 before
they deploy a product. Considering that NT 4.0 Server was at SP6 prior to
the beginning of my 52 month analysis, if you waited for SP6 of W2K3 before
deploying you may only experience an 8.333% reduction in the number of
vulnerabilities which affect it. IOWs, if you think that waiting for a
product to mature is going to make it less vulnerable, you're probably
waiting forever.

I also made the point that since newer versions aren't significantly less
vulnerable, people aren't going to upgrade to them to be less vulnerable.
That may be something MS shareholders might be concerned about, if they
believe the desire to be less vulnerable is a purchase motivator. The
actual bulletin point in the presentation was; "If Microsoft made
money from upgrades, you'd think these numbers might bother some
people"

4. I then did a comparison of desktop OS' where the browsers were kept
up-to-date, revised to new versions as they were released. It showed, to
me, that this tactic didn't significantly reduce vulnerabilities either.
Again, largely because vulnerabilities affected all versions.

5. I then did a comparison of NT 4.0 desktops, one which stuck with IE 5.01
throughout the time, another which upgraded and stuck with IE 5.5, and
another which upgraded to each new IE version. Overall, the one which stuck
with IE 5.01 had less vulnerabilities, but even so, it was only 7% less
vulnerable than the one which upgraded to each new version.

6. I made the same comparison using a Windows 2000 Professional and found
almost identical differences (albeit there were more vulnerabilities
applicable to W2K than NT 4.0). I then had a slide that simply said;
"Older is clearly better!"

Throughout my presentation the lowest number of vulnerabilities always
applied to NT 4.0. My point here was that new features meant new
vulnerabilities, either because new features were introduced which had new
vulnerabilities, or, new versions meant a new examination by vulnerability
discoverers. In retrospect, chances are the reason new versions means new
vulnerabilities is because of new examination, since by and large
vulnerabilities apply to all versions.

In the context of Microsoft's security efforts, I made comments about the
fact that the newer versions had more vulnerabilities than the older
versions. Despite the fact the difference is extremely small, fact is that
its not getting better, something which cannot be ignored by analysis of
the facts.

7. I then compared IIS versions. Given the timeframe of the products, the
numbers are very different;

IIS 4.0 = 231 vulnerabilities
IIS 5.0 = 282 vulnerabilities
IIS 6.0 = 60 vulnerabilities

I went on to say that in the period since W2K3's release, IIS 6.0 boxes
were 11% less vulnerable than W2K IIS 5.0 servers. This, however, IMO was
largely due to configuration and not a lack of vulnerable code. I said that
if you configured any IIS box the way W2K3 IIS 6.0 was configured and you'd
get roughly the same security. IOWs, where were the results of the Security
Push? Surely the results weren't only a new configuration.

8. My last graph depicted the number of vulnerabilities versus the number
of patches. IOWs, does the fact that there have been fewer patches of late
mean there have been fewer vulnerabilities. The graph indicated that the
answer is no, not really. At various points in time over the past 52 months
there have been ups and downs, and the last 6 months is no different.

9. My concluding slides were all titled "Patch-O-Mania". I
started by stating that I believed patching was not a reason to upgrade to
a new product. IOWs, if you thought you'd upgrade to a new product because
it would mean you wouldn't have to patch so often, you're wrong. I
questioned whether or not an 11% reduction in the vulnerability of W2K3
represented a start to Microsoft's Security Push. Personally, I don't
believe that's a significant enough figure to say it really is.

I then said the Security Push is about consumers, not corporate users. I
believe the MS Security Bulletins are designed for home users, not for
corporate users. The whole idea of patching is primarily intended, again
IMO, for home users. I mentioned how I believed that XP SP2 was excellent,
for home users, and that it would enable consumers to be less aware of
"Patch-O-Mania" because automatic updates would be enabled. Since
I was in Australia, and since there people pay for bandwidth above an ~50MB
cap, I did point out however that XPSP2 was 276MB and its adoption is
required for us to reap its benefits.

I concluded with Corporations. TruSecure's Sasser survey showed that unless
you achieved 100.0% patching you were in worse shape than if you didn't try
patching at all. I pointed out that none of Microsoft's patch deployment
products could guarantee 100.0% effective patch deployment, nor could any
3rd party product. Ergo, Corporations have no way of knowing whether
they've achieved the 100.0% patch deployment. I questioned how the Security
Push could think that patching was an effective part of its efforts given
these facts.

So, in the end, I wasn't recommending that people stick with NT 4.0, nor
was I saying that newer products are more exploitable or less secure than
older ones. In the numerous media interviews I gave throughout the region
over the days following the event I repeatedly told reporters this. I do
believe that we have received insufficient benefits from the Microsoft
Security Initiative, but stressed that XP SP2 represented very significant
changes in the way MS approaches security. I stressed that it remained to
be seen whether or not MS will be able to apply the new principles
contained within XP SP2 to Corporate users, but if they can it would be
great.

I called for a new SP for Windows NT 4.0, considering the fact its still
widely in use and has been more than 4 years without one yet has had at
least 68 vulnerabilities patched.

I stressed that, IMO, far too much effort is being placed on patching IE
vulnerabilities. To the best of my knowledge, only 2 wide-spread attacks
have occurred involving IE vulnerabilities, yet there have been at least 83
vulnerabilities patched for IE. Clearly a lot of effort is being spent
patching vulnerabilities which have not resulted in exploits, IMO, a large
waste of Corporate resources.

Finally, without making a sales pitch, I tried to stress that there are
many ways to mitigate against vulnerabilities. TruSecure Corporation, my
employer, and I, firmly believe that patching is amongst the least
effective methods. For example, through our knowledge transfer annual
subscription Risk Management Programs, we helped our customers stay secure
despite only recommending 3 Microsoft patches be applied urgently last
year. For more information about those services, see;
http://www.trusecure.com/solutions/programs/risk_reduction.shtml
The bottom line is that we, TruSecure Corporation, and I, are firm
believers in Microsoft Products. Anyone who's been a subscriber of
NTBugtraq for any length of time should realize this. We and Microsoft may
differ on the best approaches to stay secure, or on where Microsoft should
put its efforts or priorities, we remain a committed Microsoft Partner.
Neither TruSecure nor Microsoft have asked me to say this, I just feel that
with so many media reports making me out to be a vehement Microsoft
detractor, clarification was needed.

Cheers,
Russ - TruSecure Corporation Senior Scientist/NTBugtraq Editor (note: I've
wanted to drop the "Surgeon General" moniker for 2 years now. I
dislike "Senior Scientist" almost as much considering I never
finished High School. Suggestions for a new corporate title for me are
welcome!!)

-----

--- BBBS/NT v4.01 Flag-5