[idn] Figuring out what can be displayed (was: Re: DocumentStatus))

[John C Klensin <klensin@jck.com>]

Coming back to an old-ish thread whose implications I think I
have finally managed to explain to myself...

--On Tuesday, 03 September, 2002 02:56 +0000 "Adam M. Costello"
<idn.amc+0@nicemice.net.RemoveThisWord> wrote:

> The spec says that an application SHOULD NOT show the ACE form
> if it can correctly display the Unicode form, and it SHOULD
> show the ACE form if it cannot correctly display the Unicode
> form.  Both recommendations have equal weight.  If the
> application cannot know which Unicode characters can be
> displayed, then it cannot know which recommendation applies.
> If the application is optimistic about the display
> capabilities, it risks violating the second recommendation,
> and if the application is pessimistic, it risks violating the
> first recommendation.

In a number of, probably most, contemporary and recent operating
systems, applications actually have no clue what "they" can
display.  They organize and format a string and pass that string
to some sort of TTY or print routine, and the device driver /
display interface routines do whatever they think is appropriate
with that string.  It is only very rarely the case that the
application can query the display/TTY routines about whether a
particular character sequence can be displayed reasonably
(whatever "reasonably" means).  And, for most contemporary
operating systems, that is the way it _must_ be: the capability
of stream diversion by a calling agent or operating system or
file manipulation call (in U**x terminology, think "pipe")
implies that the display capabilities of "an application" is
unknown to the code of that application at the time it is
executing.

So, an application must, ultimately, either decide that its
operating environment can handle Unicode acceptably, convert the
ACE form to Unicode, pass that off to the display driver in some
locally-preferred form, and hope for the best.  If the operating
system and display cannot process and display Unicode, something
will, no doubt, be displayed -- some hex representation, black
blocks, etc., depending on the system and configuration settings
-- but it almost certainly won't be ACE.  Or the application
must decide that it can't know and therefore must display the
ACE form always, in which case IDNA is not a solution, or even
helpful, to the effort to provide user-level IDNs turns out to
be hopeless.

If, in order to be fully-conformant, a specification requires an
implementation to make a decision based on something it can't
know, that is, IMO (in the language of RFC 2026) a "technical
omission", and one of fairly serious proportions.  If we cannot
provide a better model, the text cannot, I think, say anything
stronger than that the application MAY show either the ACE or
the Unicode form, at its discretion, and leave this issue up to
user pressure and implementation quality.

> The situation is symmetric.  I see no reason to conclude that
> the spec favors optimism or pessimism.

I agree.  Non-discriminatory.  Screwed either way :-(

     john