From: Adam <""4thwormcastfromthemolehill\"{at}the field.near
the bridge">

Rich wrote:
>    Two different issues.  IE has to apply heuristics to file types
> because the servers that return this content often return bogus, both
> incorrect or invalid, types.  It's not easy to fix.  IE had to do this
> because netscape didn't enforce types and to be compatible IE couldn't
> either.  Because the types aren't enforced lots of servers still do this
> wrong.  Because they do types can't be enforced.
>
>    Text and Unicode is first a false distinction.  It's all text.  In
> the case of Notepad, you mean UTF-16 text vs. UTF-8 text vs. ANSI text
> as this is the distinction that Notepad makes on load.  Even that misses
> the complexity as what people call ANSI is actually any of 14 distinct
> and incompatible ANSI encodings and is often one of many OEM encodings
> which may be distinct from any of the ANSI ones.  It is complicated
> because for many of the ANSI encodings including the one used in the
> U.S. and Western Europe, anything could be valid.  Because of this it is
> not always possible to make a distinction between UTF-16 and ANSI as a
> file could validly be either.  UTF-8 is restrictive so it is easy to
> tell if something is valid UTF-8.  That could still be a problem as
> valid UTF-8 could be valid ANSI too.  Instead some heuristics are
> applied.  For example if you see 0D 00 0A 00 then the file is probably
> UTF-16 while if you see 0D 0A it may be ANSI though U+0A0D might be a
> valid Unicode character.  I didn't look.
>

Yeah hidden chars esp in XML can be a bitch.

Especially when XML chars rendered from html streams (file or other)
processed from CDATA.

Roll on XHTML.

Adam

--- BBBS/NT v4.01 Flag-5