\startcomponent co-en-08

\project manual
\product cont-en

\chapter[hyphenation,languages]{Language specific issues}

\section{Introduction}

One of the more complicated corners of \CONTEXT\ is the
department that deals with languages. Fortunately users will
seldom notice this, but each language has its own demands
and we put quite some effort in making sure that most of the
issues on hyphenation rules and accented and non latin
characters could be dealt with. For as long as it does not
violate the \CONTEXT\ user interface, we also support
existing input schemes. 

In the early days \TEX\ was very American oriented, but
since \TEX\ version~3 there is (simultaneous) support for
multiple languages. The input of languages with many accents
|<|sometimes more accents per character|>| may look rather
complicated, depending on the use of dedicated input
encodings or special \TEX\ commands.  

The situation is further complicated by the fact that
specific input does not have a one||to||one relation with
the position of a glyph in a font. We discussed this in \in
{section}[encoding]. It is important to make the right
choices for input and font encoding. 

In this chapter we will deal with hyphenation and language
specific labels. More details can be found in the language
definition files (\type {lang-<<xxx>>}), the font files
(\type{font-<<xxx>>}) and the encoding files (\type
{enco-<<xxx>>}). There one can find details on how to define
commands that deal with accents and special characters as
covered in a previous chapter, sorting indexes, providing
support for \UNICODE, and more. 

\section{Automatic hyphenating}
\index{languages}
\index{hyphenation}
\macro{\tex{taal}}
\macro{\tex{nl}}
\macro{\tex{en}}
\macro{\tex{fr}}
\macro{\tex{de}}
\macro{\tex{sp}}

Each language has its own hyphenation rules. As soon as you
switch to another language, \CONTEXT\ will activate the
appropriate set of hyphenation patterns for that language.
Languages are identified by their official two character
identifiers, like: Dutch (\type{nl}), English (\type{en}),
German (\type{de}) and French (\type{fr}). A language is
chosen with the following command: \footnote{In case of any
doubt please check if the hyphenation patterns are included
in the \type {fmt}||file.}

\showsetup{language}

Some short cut  commands are also available. They can be used
enclosed in braces: 

\startexample
\starttyping
\nl  \en  \de  \fr  \sp  \uk  \pl  \cz  ...
\stoptyping
\stopexample

The command \type {\language[nl]} can be compared with
\type {\nl}. The first command is more transparant. The two
character commands may conflict with existing commands.
Take, for example, Italian and the code for {\it italic}
type setting. For this reason we use capitals for commands
that may cause any conflicts. One may also use the full 
names, like \type {czech}. 

At any instance you can switch to another language. In the
example below we switch from English to French and vice
versa. 

\startexample
\typebuffer[messian]
\stopexample

We use these language switching commands if we cannot be
certain that an alternative hyphenation pattern is
necessary. 

\startcolumns[n=4,,rule=on]
\getbuffer[messian]
\stopcolumns

How far do we go in changing languages. Borrowed words like
perestrojka and glasnost are often hyphenated okay, since
these are Russian words used in an English context. When
words are incorrectly hyphenated you can define an
hyphenation pattern with the \TEX||command: 

\startexample
\starttyping
\hyphenation{<<ab-bre-via-tion>>}
\stoptyping
\stopexample

You can also influence the hyphenation in a text by 
indicating the allowed hyphenation pattern in the word:
at the right locations the command \type{\-}
is added: \type{al}\type{\-}\type{lo}\type{\-}\type{wed}.

\section{Definitions and setups}
\macro{\tex{installlanguage}}
\macro{\tex{setuplanguage}}

When a format file is generated the hyphenation pattern one
needs should be added to this file. The definition and
installation of a language is therefore not transparant for
the user. We show the process to give some insight in the
mechanism. An example: \footnote {The somewhat strange name
\type {\upperleftsinglesixquote} is at least telling us what
the quote will look like.} 
 
\starttyping
\installlanguage
  [en]
  [spacing=broad,
   leftsentence=---,
   rightsentence=---,
   leftsubsentence=---,
   rightsubsentence=---,
   leftquote=\upperleftsinglesixquote,
   rightquote=\upperrightsingleninequote,
   leftquotation=\upperleftdoublesixquote,
   rightquotation=\upperrightdoubleninequote,
   date={month,\ ,day,{,\ },year},
   default=en,
   state=stop] 
\stoptyping

and:

\starttyping
\installlanguage
  [uk]
  [default=en,
   state=stop]
\stoptyping

With the first definition you define the language component.
You can view this definition in the file \type
{lang-ger.tex}, the german languages. Languages are arranged
in language groups. This arrangement is of no further
significance at the moment. Since language definitions are 
preloaded, users should not bother about setting up such 
files. 

The second definition inherits its set up from the English
installation. In both definitions \type {state} is set at
\type {stop}. This means that no patterns are loaded yet.
That is done in the files \type{cont-<<xx>>}, the language
and interface specific \CONTEXT\ versions. As soon as \type
{state} is set at \type {start}, a new pattern is loaded,
which can only be done during the generation of a format
file. 

We use some conventions in the file names of the patterns
\type {lang-xx.pat} and the exceptions \type {lang-xx.hyp}.
Normally a language is installed with a two character code.
However there are three character codes, like {deo} for
hyphenating \quote {old deutsch} and \type {nlx} the Dutch
extended characterset, or 8||bit encoding. On distributions 
that come with patterns, the filenames mentioned can be mapped 
onto the ones available on the system. This happens in the 
file \type {cont-usr.tex}.  

After installation you are not bound to the two character
definitions. Default the longer (English) equivalents are
defined: 

\starttyping
\installlanguage[german][de]
\stoptyping

\showsetup{installlanguage}

\showsetup{setuplanguage}

The setup in these commands relate to the situations that are
shown below. 

\startbuffer
\currentdate
|<|all right there we go|>|
|<| |<|all right|>| there we go|>|
|<|all right |<|there|>| we go|>|
\quote{all right there we go}
\quotation{all right there we go}
\quotation{\quote{all right} there we go}
\quotation{all right \quote{there} we go}
\stopbuffer

\typebuffer

This becomes: 

\startlines
\getbuffer 
\stoplines

We will discuss \type{||} in one of the next sections.

\section[datum]{Date}
\index{date}
\macro{\tex{currentdate}}
\macro{\tex{date}}

Typesetting a date is also language specific so we have to
pay some attention to dates here. When the computer runs at
the actual time and date the system date can be recalled
with: 

\showsetup{currentdate}

The sequence in which \type {day}, \type {month} and \type
{year} are given is not mandatory. The pattern \type
{[day,month,year]} results in \currentdate [day,month,year].
We use \type {\currentdate[weekday,month,day,{,},year]} to
obtain \currentdate[weekday,month,day,{,},year]. 

A short cut looks like this: \type {[dd,mm,yy]} and will
result in \currentdate [dd,mm,yy]. Something like \type
{[d,m,y]} would result in \currentdate [d,m,y] and with
\type {[referral]} you will get a \currentdate [referral].
Combinations are also possible. Characters can also be added
to the date pattern. The date \currentdate [dd,--,mm,--,yy] is
generated by the pattern \type {[dd,--,mm,--,yy]}. 

A date can be (type|)|set with the command:

\showsetup{date}

The first (optional) argument is used to specify the date: 

\startbuffer
\date[d=10,m=3,y=1996][weekday,month,day, year]
\stopbuffer

\startexample
\typebuffer
\stopexample

When no argument is given you will obtain the actual date.
When the second argument is left out the result equals that
of \type {\currentdate}. The example results in: 

\startreality
\getbuffer
\stopreality

\section[labels]{Labels and heads}
\index{labels}
\index{heads}
\macro{\tex{mainlanguage}}
\macro{\tex{setuplabeltext}}
\macro{\tex{setupheadtext}}
\macro{\tex{labeltext}}
\macro{\tex{headtext}}

In some cases \CONTEXT\ will generate text labels
automatically, for example the word {\bf Figure} is
generated automatically when a caption is placed under a
figure. These kind of words are called textlabels. Labels
can be set with the command: 

\showsetup{setuplabeltext}

Relevant labels are: \type {table}, \type {figure}, \type
{chapter}, \type {appendix} and comparable text elements. An
example of such a set up is: 

\startexample
\starttyping
\setuplabeltext[en][chapter=Chapter ]
\setuplabeltext[nl][hoofdstuk=Hoofdstuk ]
\stoptyping
\stopexample

The space after \type{Chapter} is essential, because
otherwise the chapternumber will be placed right after the
word Chapter (Chapter1 instead of Chapter~1). A labeltext
can recalled with: 

\showsetup{labeltext}

Some languages, like Chinese, use split labels. These can be 
passed as a comma separated list, like  \type
{chapter={left,right}}. 

Titleheads for special sections of a document, like
abbreviations and appendices are set up with: 

\showsetup{setupheadtext}

Examples of titleheads are \type{Content}, \type{Tables},
\type{Figures}, \type{Abbreviations}, \type{Index} etc. An
example definition looks like: 

\startexample
\starttyping
\setupheadtext[content=Content]
\stoptyping
\stopexample

A header can be recalled with:

\showsetup{headtext}

Labels and titleheads are defined in the file \type
{lang-<<xxx>>}. You should take a look in these files to
understand the use of titleheads and labels. 

The actual language that is active during document
generation does not have to be the same language that is
used for the labels. For this reason next to \type
{\language} we have: 

\showsetup{mainlanguage}

When typesetting a document, there is normally one main
language, say \type {\mainlanguage[en]}. A temporary switch
to another language is then accomplished by for instance
\type {\language[nl]}, since this does not influence the
labels and titles. language. 

\section{Language specific commands}
\index{german}

German \TEX\ users are accustomed to entering \type {"e} and
getting \"e typeset in return. This and a lot more are
defined in \type {lang-ger} using the compound character
mechanism built in \CONTEXT. Certain two or three character
combinations result in one glyph or proper hyphenation. The
example below illustrates this. Some macros are used that
will not be explained here. Normally, users can stick to 
simply using the already defined commands. 

\starttyping
\startlanguagespecifics[de]
  \installcompoundcharacter "a  {\moveaccent{-.1ex}\"a\midworddiscretionary}
  \installcompoundcharacter "s  {\SS}
  .....
  \installcompoundcharacter "U  {\smashaccent\"U}
  \installcompoundcharacter "Z  {SZ}
  .....
  \installcompoundcharacter "ck {\discretionary {k-}{k}{ck}}
  \installcompoundcharacter "TT {\discretionary{TT-}{T}{TT}}
  .....
  \installcompoundcharacter "`  {\handlequotation\c!leftquotation}
\stoplanguagespecifics
\stoptyping

The command \type {\installcompoundcharacter} takes care of
the German type setting, \type {"a} is converted to {\de
"a}, \type {"U} in {\de "U}, \type {"ck} for the right
hyphenation, etc. One can add more definitions, but this will
violate portability. In a Polish \CONTEXT\ the \type {/} is 
used instead of a \type {"}. 

\section{Automatic translation}
\index{translate}
\macro{\tex{translate}} 

It is possible to translate a text automatically in the
actual language. This may be comfortable when typesetting
letterheads. The example below illustrates this. 

\showsetup{translate}

\startbuffer
It depends on the actual language whether a labeltext is type set in 
English {\en as an \translate [en=example, fr=exemple], \fr or in French
as an \translate}.
\stopbuffer

\startexample
\typebuffer
\stopexample

The second command call \type{\translate} uses the applied
values. That is, \type {\translate} with no options uses the 
options of the last call to \type {\translate}.  

\startreality
\getbuffer
\stopreality

\section{Composed words}
\index{hyphen}
\macro{\tex{setuphyphenmark}}

Words consisting of two separate words are often separated
by an intra word dash, as in x||axis. This dash can be
placed between \type{| |}, for example \type{|--|}. This
command, which does not begin with a \texescape, serves
several purposes. When \type{||} is typed the default intra
word dash is used, which is \type{--}. This dash is set up
with: 

\showsetup{setuphyphenmark}

The \type {| |} is also used in word combinations like
(intra)word, which is typed as \type {(intra|)|word}. The
mechanism is not foolproof but it serves most purposes. In
case the hyphenation is incorrect you can hyphenate the
first word of the composed one by hand: \type
{(in\-tra|)|word}. 

\placetable
  {Hyphenation of composed words.}
  {\getbuffer[hyph-1]}

The main reason behind this mechanism is that \TEX\ doesn't
really know how to hyphenate composed words and how to
handle subsentences. \TEX\ know a lot about math, but far
less about normal texts. Using this command not only serves
consistency, but also makes sure that \TEX\ can break
compound words at the right places. It also keeps boundary
characters at the right place when a breakpoint is inserted. 

\stopcomponent

