Subject: The future of NetBSD
To: None 
From: Charles M. Hannum 
List: netbsd-users
Date: 08/30/2006 19:27:23

The NetBSD Project has stagnated to the point of irrelevance.  It has gotten to 
the point that being associated with the project is often more of a liability 
than an asset.  I will attempt to explain how this happened, what the current 
state of affairs is, and what needs to be done to attempt to fix the situation.

As one of the 4 originators of NetBSD, I am in a fairly unique position. I am 
the only one who has continuously participated and/or watched the project over 
its entire history.  Many changes have taken place, and at the same time many 
things have remained the same -- including some of our early mistakes.

I'd like to say that I'm some great visionary, who foresaw the whole OSS 
market, but the fact is that's BS.  When we started the project, Linux and 
386BSD were both little hobbyist systems, both pretty buggy, and both lacking a 
lot of important hardware support.  Mostly we were scratching an itch: there 
was no complete package of 386BSD plus the necessary patches to make it run on 
more systems and fix bugs, and there was no sign that Bill Jolitz was going to 
resurface and do anything.

Much of the project structure evolved because of problems we had early on.  
Probably our best choice was to start using central version control right off; 
this has enabled a very wide view of the code history and (eventually) made 
remote collaboration with a large number of developers much easier.  Some other 
things we fudged; e.g. Chris got tired of being the point man for everything, 
and was trying to graduate college, so we created an internal "cabal" for 
managing the project, which became known as the "core group". 
Although the web 
was very new, we set up a web site fairly early, to disseminate information 
about the project and our releases.

Much of this early structure (CVS, web site, cabal, etc.) was copied verbatim 
by other open source (this term not being in wide use yet) projects -- even the 
form of the project name and the term "core".  This later became a kind of 
standard template for starting up an open source project.

Unfortunately, we made some mistakes here.  As we've seen over the years, one 
of the great successes of Linux was that it had a strong leader, who set goals 
and directions, and was able to get people to do what he wanted -- or find 
someone else to do it.  This latter part is also a key element; there was no 
sense that anyone else "owned" a piece of Linux (although de
facto "ownership" 
has happened in some parts); if you didn't produce, Linus would use someone 
else's code.  If you wanted people to use your stuff, you had to keep moving.

NetBSD did not have this.  Partly due to lack of people, and partly due to a 
more corporate mentality, projects were often "locked".  One
person would say 
they were working on a project, and everyone else would be told to refer to 
them.  Often these projects stagnated, or never progressed at all.  If they 
did, the motivators were often very slow. As a result, many important projects 
have moved at a glacial pace, or never materialized at all.

I'm sorry to say that I helped create this problem, and that most of the 
projects which modeled themselves after NetBSD (probably due to its high 
popularity in 1993 and 1994) have suffered similar problems.  FreeBSD and 
XFree86, for example, have both forked successor projects (Dragonfly and X.org) 
for very similar reasons.

Unfortunately, these problems still exist in the NetBSD project today, and 
nothing is being done to fix them.

--

I won't attempt to pin blame on any specific people for this, except to say 
that some of it is definitely my fault.  It's only in retrospect that I see so 
clearly the need for a very strong leader.  Had I pursued it 10 years ago, 
things might be very different.  Such is life.  But let's talk about the 
situation today.

Today, the project is run by a different cabal.  This is the result of a coup 
that took place in 2000-2001, in which The NetBSD Foundation was taken over by 
a fraudulent change of the board of directors.  (Note: It's probably too late 
for me to pursue any legal remedy for this, unfortunately.)  Although "The 
NetBSD Project" and "The NetBSD Foundation" were intended
from the start to be 
separate entities -- the latter supplying support infrastructure for the former 
-- this distinction has been actively blurred since, so that the current 
"board" of TNF has rather tight control over many aspects of TNP.

Were TNF comprised of a good set of leaders, this situation might be somewhat 
acceptable -- though certainly not ideal.  The problem is, there are really no 
leaders at this point.  "Goals" for releases are not based on
customer feedback 
or looking forward to future needs, but solely on the basis of what looks like 
it's bubbled up enough that it might be possible to finish in time.  There is 
no high-level direction; if you ask "what about the problems with
threads" or 
"will there be a flash-friendly file system", the best you'll get
is "we'd love 
to have both" -- but no work is done to recruit people to code these things, or 
encourage existing developers to work on them.

This vacuum has contributed materially to the project's current stagnation.  
Indeed, NetBSD is very far behind on a plethora of very important projects.  
Threading doesn't really work across multiple CPUs
-- and is even somewhat buggy on one CPU.  There is no good flash file
system.  There is no file system journaling (except for LFS, which is still 
somewhat experimental).  Although there's been some recent work on suspend 
support, it's still mostly broken.  Power management is very primitive.  Etc.  
Even new hardware support is generally not being originated in NetBSD any more; 
it's being developed by FreeBSD and OpenBSD, and being picked up later.  (I 
think the only recent exception to this of any significance is Bluetooth 
support.)

For these reasons and others, the project has fallen almost to the point of 
irrelevance.  (Some people will probably argue that it's beyond that point, but 
I'm trying to be generous.)  This is unfortunate, especially since NetBSD usage 
-- especially in the embedded space -- was growing at a good rate in 2000 and 
2001, prior to the aforementioned coup.

--

At this point most readers are probably wondering whether I'm just writing a 
eulogy for the NetBSD project.  In some ways, I am -- it's clear that the 
project, as it currently exists, has no future.  It will continue to fall 
further behind, and to become even less relevant.  This is a sad conclusion to 
a project that had such bright prospects when it started.

I admit that I may be wrong about this, but I assume that most people who have 
contributed to NetBSD, and/or continue to do so, do not desire to see the 
project wallow away like this.  So I will outline what I think is the only way 
out:

1) There must be a strong leadership, and it is not the current one.
   The leadership must honestly want NetBSD to be a premier, world class
   system with leading edge features.  The leadership must set
   aggressive goals, and actively recruit people to make them happen.

2) There must be no more "locking" of projects.  Just because one person
   is supposedly working on a problem, that doesn't mean you shouldn't.
   If there ideas are dumb, or even just suboptimal, do it better!  If
   there is no progress, hop on it.  Don't wait for someone else.

3) The project must become an *actual* meritocracy, not what I call a
   "volumetocracy".  Right now, the people who exert the most influence
   are often the people who produce the least useful product.  Indeed,
   they are often people who produce little more than fluff (e.g.
   changing line-ending whitespace!), and often break things.

4) Speaking of which, there must be negative feedback to discourage
   people from breaking stuff.  This has been a continual problem with
   certain "developers" for more than a decade.

5) There are a number of aspects of the NetBSD architecture that are
   flat out broken, and need serious rehabilitation.  Again, the
   leadership needs to recruit people to do these things.  Some of them
   include:

   * serious problems with the threading architecture (including the
     user-kernel interface), as mentioned earlier;
   * terrible support for kernel modules;
   * the horrible mess that is 32/64-bit compatibility, resulting in
     32-bit apps often not working right on 64-bit kernels; and
   * unbounded maintenance work due to inappropriate and rampant use of
     "quirk" tables and chip-specific tables; e.g. in SCSI, ATAPI, IDE,
     ACPI and SpeedStep support.  (I actually did much of this work for
     SCSI, but am not currently able to commit it.)

6) The existing NetBSD Foundation must be disbanded, and replaced with
   an organization that fulfills its original purpose: to merely handle
   administrative issues, and not to manage day-to-day affairs.  The
   extra committees, which mostly do nothing, must be disbanded -- they
   serve only to obfuscate things.  Everything else must revert to the
   historically separate entity, the NetBSD Project, to be managed based
   on technical merits.  There must be no perceived glamour in
   participating in the Foundation; it must be composed of people doing
   it because they are dedicated and want to help the project.

   (I will note here that this is not due to bitterness over the coup.
   Keeping the NetBSD Project as an unincorporated association actually
   helps protect it.)

7) The "core" group must be replaced with people who are actually
   competent and dedicated enough to review proposals, accept feedback,
   and make good decisions.  More to the point, though, the "core" group
   must only act when *needed* -- most technical decisions should be
   left to the community to hash out; it must not preempt the community
   from developing better solutions.  (This is how the "core" group
   worked during most of the project's growth period.)

8) There must be a set of commit standards -- e.g. about when it is or
   is not acceptable to commit changes that do not change functionality;
   when multiple changed must be batched in one commit; etc.  Right now
   it is difficult to sort the wheat from the chaff.  In addition, there
   must be standards of review.

I must repeat a point I've made earlier.  The current "management" of the 
project is not going to either fix the project's problems, or lead the project 
to solutions.  They are going to maintain the status quo, and nothing else.  If 
the project is to rise from its charred stump, this "management" must be 
disbanded and replaced wholesale.  Anything less is a non-solution.

--

To some of you, I would like to apologize.  There *are* NetBSD developers doing 
good work even now.  I'd like to particularly recognize and thank those working 
on kernel locking and UVM problems; wireless support (though I'm not sure what 
happened to my extensive set of rtw bug fixes); Bluetooth; G5; and improved ARM 
support.  This is all good stuff.  In the bigger picture, though, the project 
needs to do a lot more.

--
- Charles Hannum - past founder, developer, president and director of
  The NetBSD Project and The NetBSD Foundation; sole proprietor of The
  NetBSD Mission; proprietor of The NetBSD CD Project.

[I'm CCing this to FreeBSD and OpenBSD lists in order to share it with the 
wider *BSD community, not to start a flame war.  I hope that people reading it 
have the tact to be respectful of their peers, and consider how some of these 
issues may apply to them as well.]


--- BBBS/NT v4.00 MP