% language=uk

\startcomponent hybrid-ebooks

\environment hybrid-environment

\startchapter[title={E-books: Old wine in new bottles}]

\startsection [title={Introduction}]

When Dave Walden asked me if \TEX\ (or \CONTEXT) can generate
ebooks we exchanged a couple of mails on the topic. Although I had
promised myself never to fall into the trap of making examples for
the sake of proving something I decided to pick up an experiment
that I had been doing with a manual in progress and look into the
\HTML\ side of that story. After all, occasionally on the
\CONTEXT\ list similar questions are asked, like \quotation {Can
\CONTEXT\ produce \HTML ?}. \footnote {This text appeared in the
\EUROTEX\ 2011 proceedings.}

\stopsection

\startsection [title={Nothing new}]

When you look at what nowadays is presented as an ebook document,
there is really not that much new going on. Of course there are
very advanced and interactive documents, using techniques only
possible with recent hardware and programs, but the average ebook
is pretty basic. This is no surprise. When you take a novel, apart
from maybe a cover or an occasional special formatting of section
titles, the content is typeset pretty straightforward. In fact,
given that formatters like \TEX\ have been around that can do such
jobs without much intervention, it really takes some efforts to
get that job done badly. It was kind of shocking to notice that on
one of the first e-ink devices that came available the viewing was
quite good, but the help document was just some word processor
output turned into bad looking \PDF. The availability of proper
hardware does not automatically trigger proper usage.

I can come up with several reasons why a novel, published as an
ebook does not look much more interesting and in many cases looks
worse. First of all it has to be produced cheap, because there is
also a printed version and because the vendor of some devices also
want to make money on it (or even lock you into their technology
or shop). Then, it has to be rendered on various devices so the
least sophisticated one sets the standard. As soon as it gets
rendered, the resolution is much worse than what can be achieved
in print, although nowadays I've seen publishers to go for quick
and dirty printing, especially for reprints.

Over a decade ago, we did some experiments with touch screen
computers. They had a miserable battery live, a slow processor and
not much memory, but the resolution was the same as on the now
fashionable devices. They were quite suitable for reading but even
in environments where that made sense (for instance where huge
manuals had to be carried around) using such devices never took
off. Nowadays we have wireless access and usb sticks and memory
cards to move files around, which helps a lot. And getting a
quality on it comparable to what can be done today was no big
deal, at least not from the formatting point of view.

If you look in the context you will find quite some presentation
styles that can serve as basis for an ebook style. Also some of
the \CONTEXT\ manuals come with two versions: one for printing and
one for viewing on the screen. A nice example is the \METAFUN\
manual (see \in {figure} [fig:metafun]) where each page has a
different look.

\placefigure
  [here]
  [fig:metafun]
  {An example page from the metafun manual.}
  {\externalfigure[ebook-metafun-2.png][width=\textwidth]}

It must be said that the (currently only black and white) devices
that use electronic ink have a perceived resolution that is higher
than their specifications which is due to the semi analogue way
the \quote {ink} behaves. In a similar fashion clever
anti|-|aliasing can do wonders on lcd screens. On the other hand
they are somewhat slow and a display refresh is not that
convenient. Their liquid crystal counterparts are much faster but
they can be tiresome to look at for a long time and reading a book
on it sitting in the sun is a no|-|go. Eventually we will get
there and I'm really looking forward to seeing the first device
that will use a high resolution electrowetting \CMYK\ display.
\footnote {http://www.liquavista.com/files/LQV0905291LL5-15.pdf}
But no matter what device is used, formatting something for it is
not the most complex task at hand.

\stopsection

\startsection[title={Impact}]

Just as with phones and portable audio devices, the market for
tablets and ebook|-|only devices evolves rapidly. While writing
this, at work I have one ebook device and one tablet. The ebook
device is sort of obsolete because the e-ink screen has
deteriorated even without using it and it's just too slow to be
used for reference manuals. The tablet is nice, but not that
suitable for all circumstances: in the sun it is unreadable and at
night the backlight is rather harsh. But, as I mentioned in the
previous section, I expect this to change.

If we look at the investment, one needs good arguments to buy
hardware that is seldom used and after a few years is obsolete.
Imagine that a family of four has to buy an ebook device for each
member. Add to that the cost of the books and you quickly can end
up with a larger budget than for books. Now, imagine that you want
to share a book with a friend: will you give him or her the
device? It might be that you need a few more devices then. Of
course there is also some data management needed: how many copies
of a file are allowed to be made and do we need special programs
for that? And if no copy can be made, do we end up swapping
devices? It is hard to predict how the situation will be in a few
years from now, but I'm sure that not everyone can afford this
rapid upgrading and redundant device approach.

A friend of mine bought ebook devices for his children but they
are back to paper books now because the devices were not
kid|-|proof enough: you can sit on a book but not on an ebook
reader.

The more general devices (pads) have similar problems. I was
surprised to see that an iPad is a single user device. One can of
course hide some options behind passwords but I'm not sure if
parents want children to read their mail, change preferences,
install whatever app they like etc. This makes pads not that
family friendly and suggests that such a personal device has to be
bought for each member. In that case it suddenly becomes a real
expensive adventure. So, unless the prices drop drastically, pads
are no valid large scale alternative for books yet.

It might sound like I'm not that willing to progress, but that's
not true. For instance, I'm already an enthusiastic user of a
media player infrastructure. \footnote {The software and hardware
was developed by SlimDevices and currently is available as
Logitech Squeezeserver. Incidentally I can use the iPad as
advanced remote control.} The software is public, pretty useable,
and has no vendor lock|-|in. Now, it would make sense to get rid
of traditional audio media then, but this is not true. I still buy
\CD's if only because I then can rip them to a proper lossless
audio format (\FLAC). The few \FLAC s that I bought via the
internet were from self|-|publishing performers. After the
download I still got the \CD's which was nice because the booklets
are among the nicest that I've ever seen.

Of course it makes no sense to scan books for ebook devices so for
that we depend on a publishing network. I expect that at some
point there will be proper tools for managing your own electronic
books and in most cases a simple file server will do. And the most
open device with a proper screen will become my favourite. Also, I
would not be surprised if ten years from now, many authors will
publish themselves in open formats and hopefully users will be
honest enough to pay for it. I'm not too optimistic about the
latter, if only because I observe that younger family members
fetch everything possible from the internet and don't bother about
rights, so we definitely need to educate them. To some extend
publishers of content deserve this behaviour because more than I
like I find myself in situations where I've paid some 20 euro for
a \CD\ only to see that half a year later you can get it for half
the price (sometimes it also happens with books).

Given that eventually the mentioned problems and disadvantages
will be dealt with, we can assume that ebooks are here and will
stay forever. So let's move on to the next section and discuss the
look and feel of them.

\stopsection

\startsection[title={Interactivity}]

The nice thing about a paper book is that it is content and
interface at the same time. It is clear where it starts and ends
and going from one page to another is well standardized. Putting a
bookmark in it is easy as you can fall back on any scrap of paper
laying around. While reading you know how far you came and how
much there is to come. Just as a desktop on a desktop computer
does not resemble the average desktop an ebook is not a book. It
is a device that can render content either in a given or in a more
free form way.

However, an electronic book needs an interface and this is also
where at the moment it gets less interesting. Of course the
internet is a great place to wander around and a natural place to
look for electronic content. But there are some arguments to give
for buying them at a bookshop, for instance because you see a lot
(potentially) new books, often organized in topics in one glance.
It's a different way of selecting. I'm not arguing that the
internet is a worse place, but there is definitely a difference:
more aggressive advertisements, unwanted profiling that can narrow
down what is presented are a few.

Would I enter a bookshop if on the display tables there were
stacks of (current) ebook devices showing the latest greatest
books? I can imagine that at some point we have ebook devices that
have screens that run from edge to edge and then we get back some
of the appeal of book designs. It is that kind of future devices
that we need to keep in mind when we design electronic documents,
especially when after some decades we want them to be as
interesting as old books can be. Of course this is only true for
documents that carry the look and feel of a certain time and place
and many documents are thrown away. Most books have a short
lifespan due to the quality of the paper and binding so we should
not become too sentimental about the transition to another medium.

Once you're in the process of reading a book not much interfacing
is needed. Simple gestures or touching indicated areas on the page
are best. For more complex documents the navigation could be part
of the design and no screen real estate has to be wasted by the
device itself. Recently I visited a school related exhibition and
I was puzzled by the fact that on an electronic schoolboard so
much space was wasted on colorful nonsense. Taking some 20\% off
each side of such a device brings down the effective resolution to
600 pixels so we end up with 10 pixels or less per character
(showed at about 1 cm width). At the same exhibition there were a
lot of compensation programs for dyslexia advertised, and there
might be a relationship.

\stopsection

\startsection[title={Formatting}]

So how important is the formatting? Do we prefer reflow on demand
or is a more frozen design that suits the content and expresses
the wish of the author more appropriate? In the first case \HTML\
is a logical choice, and in the second one \PDF\ makes sense. You
design a nice \HTML\ document but at some point the reflow gets in
the way. And yes, you can reflow a \PDF\ file but it's mostly a
joke. Alternatively one can provide both which is rather trivial
when the source code is encoded in a systematic way so that
multiple output is a valid option. Again, this is not new and
mostly a matter of a publishers policy. It won't cost more to
store in neutral formats and it can already be done cheap for a
long time.

Somewhat interfering in this matter is digital rights management.
Where it is rather custom to buy a book and let friends or family
read the same book, it can get complicated when content becomes
bound to one (or a few) devices. Not much sharing there, and in
the worst case, no way to move your books to a better device. Each
year in the Netherlands we have the book|-|week event and
bookshops give away a book specially written for the occasion.
This year the book was also available as ebook, but only via a
special code that came with the book. I decided to give it a try
and ended up installing a not working application i.e.\ I could
not get it to load the book from the internet, and believe me, I
have a decent machine and the professional \PDF\ viewer software
that was a prerequisite.

\stopsection

\startsection[title={Using \TEX}]

So, back to Dave's question if \CONTEXT\ can generate ebooks in
the \EPUB\ format. Equally interesting is the question if \TEX\
can format an \EPUB\ file into a (say) \PDF\ file. As with much
office software, an \EPUB\ file is nothing more than a zip file
with a special suffix in which several resources are combined. The
layout of the archive is prescribed. However, by demanding that
the content itself is in \HTML\ and by providing a stylesheet to
control the renderer, we don't automatically get properly tagged
and organized content. When I first looked into \EPUB, I naively
assumed that there was some well defined structure in the content
but was surprised that this is not the case.

Let's start with answering the second question. Yes, \CONTEXT\ can
be used to convert an \EPUB\ file into a \PDF\ file. The natural
follow up question is if it can be done automatically and then
some more nuance is needed: it depends. If you download the \EPUB\
\quotation {A tale of two cities} from Charles Dickens from the
Gutenberg Project website and look into a chapter you will see
this:

\starttyping
<h1 id="pgepubid00000">A TALE OF TWO CITIES</h1>
<h2 id="pgepubid00001">A STORY OF THE FRENCH REVOLUTION</h2>
<p><br/></p>
<h2>By Charles Dickens</h2>
<p><br/>
<br/></p>
<hr/>
<p><br/>
<br/></p>
<h2 id="pgepubid00002">Contents</h2>
\stoptyping

What follows is a tables of contents formatted using \HTML\ tables
and after that

\starttyping
<h2 id="pgepubid00004">I. The Period</h2>
\stoptyping

So, a level two header is used for the subtitle of the book as
well as a regular chapter. I must admit that I had to go on the
internet to find this snippet as I wanted to check its location.
On my disk I had a similar file from a year ago when I first
looked into \EPUB. There I have:

\starttyping
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
  <head>
    <title>I | A Tale of Two Cities</title>
    ....
  </head>
  <body>
    <div class="body">
      <div class="chapter">
        <h3 class="chapter-title">I</h3>
        <h4 class="chapter-subtitle">The Period</h4>
\stoptyping

I also wanted to make sure if the interesting combination of third
and fourth level head usage was still there but it seems that
there are several variants available. It is not my intention to
criticize the coding, after all it is valid \HTML\ and can be
rendered as intended. Nevertheless, the first snippet definitely
looks worse, as it uses breaks instead of \CSS\ spacing directives
and the second wins on obscurity due to the abuse of the head
element.

These examples answer the question about formatting an arbitrary
\EPUB\ file: \quotation {no}. We can of course map the tagging to
\CONTEXT\ and get pretty good results but we do need to look at
the coding.

As such books are rather predictable it makes sense to code them
in a more generic way. That way generic stylesheets can be used to
render the book directly in a viewer and generic \CONTEXT\ styles
can be used to format it differently, e.g.\ as \PDF.

Of course, if I were asked to set up a workflow for formatting
ebooks, it would be relatively easy to do that. For instance the
Gutenberg books are available as raw text and that can be parsed
to some intermediate format or with \MKIV\ interpreted directly.

Making a style for a specific instance, like the Dickens book, is
not that complex either. After all, the amount of encoding is
rather minimal and special bits and pieces like a title page needs
some special design anyway. The zipped file can be processed
directly by \CONTEXT, but this mostly a convenience.

As \EPUB\ is just a wrapper, the next question is if \CONTEXT\ can
produce some kind of \HTML\ and the answer to that question is
positive. Of course this only makes sense when the input is a
\TEX\ source, and we have argued before that when multiple output
is needed the user might consider a different starting point.
After all, \CONTEXT\ can deal with \XML\ directly.

The main advantage of coding in \TEX\ is that the source remains
readable and for some documents it's certainly more convenient
anyway, like manuals about \TEX. In the reference manual \quote
{\CONTEXT\ \LUA\ Documents} there are the following commands:

\starttyping
\setupbackend
  [export=yes]

\setupinteraction
  [title=Context Lua Documents,
   subtitle=preliminary version,
   author=Hans Hagen]
\stoptyping

At the cost of at most 10\% extra runtime an \XML\ export is
generated in addition to the regular \PDF\ file. Given that you
have a structured \TEX\ source the exported file will have a
decent structure as well and you can therefore transform the file
into something else, for instance \HTML. But, as we already have a
good looking \PDF\ file, the only reason for having a \HTML\ as
well, is that we can reflow it. Of course wrapping up the \HTML\
into an \EPUB\ structure is not that hard. We can probably even
get away from wrapping because we have a single self contained
file.

\placefigure
  {A page from the the \PDF\ variant of the \CLD\ manual.}
  {\externalfigure[ebook-pdf.png][width=\textwidth]}

The \type {\setupbackend} command used in the \CLD\ manual has a few
more options:

\starttyping
\setupbackend
  [export=cld-mkiv-export.xml,
   xhtml=cld-mkiv-export.xhtml,
   css={cld-mkiv-export.css,mathml.css}]
\stoptyping

We explicitly name the export file and in addition specify a
stylesheet and an alternative \XHTML\ file. If you can live
without hyperlinks the \XML\ file combined with the cascading
style sheet will do a decent job on controlling the formatting.

In the \CLD\ manual chapters are coded like this:

\starttyping
\startchapter[title=A bit of Lua]

\startsection[title=The language]
\stoptyping

The \XML\ output of this becomes:

\starttyping
<division detail='bodypart'>
  <section detail='chapter' location='aut:3'>
    <sectionnumber>1</sectionnumber>
    <sectiontitle>A bit of Lua</sectiontitle>
    <sectioncontent>
      <section detail='section'>
        <sectionnumber>1.1</sectionnumber>
        <sectiontitle>The language</sectiontitle>
        <sectioncontent>
\stoptyping

The \HTML\ version has some extra elements:

\starttyping
<xhtml:a name="aut_3">
  <section location="aut:3" detail="chapter">
\stoptyping

The table of contents and cross references have \type {xhtml:a}
elements too but with the \type {href} attribute. It's interesting
to search the web for ways to avoid this but so far no
standardized solution for mapping \XML\ elements onto hyperlinks
has been agreed upon. In fact, getting the \CSS\ mapping done was
not that much work but arriving at the conclusion that (in 2011)
these links could only be done in a robust way using \HTML\ tags
took more time. \footnote {In this example we see the reference
\type {aut:3} turned into \type {aut_1}. This is done because some
browsers like to interpret this colon as a url.} Apart from this
the \CSS\ has enough on board to map the export onto something
presentable. For instance:

\starttyping
sectioncontent {
    display: block ;
    margin-top: 1em ;
    margin-bottom: 1em ;
}

section[detail=chapter], section[detail=title] {
    margin-top: 3em ;
    margin-bottom: 2em ;
}

section[detail=chapter]>sectionnumber {
    display: inline-block ;
    margin-right: 1em ;
    font-size: 3em ;
    font-weight: bold ;
}
\stoptyping

As always, dealing with verbatim is somewhat special. The following
code does the trick:

\starttyping
verbatimblock {
    background-color: #9999FF ;
    display: block ;
    padding: 1em ;
    margin-bottom: 1em ;
    margin-top: 1em ;
    font-family: "Lucida Console", "DejaVu Sans Mono", monospace ;
}

verbatimline {
    display: block ;
    white-space: pre-wrap ;
}

verbatim {
    white-space: pre-wrap ;
    color: #666600 ;
    font-family: "Lucida Console", "DejaVu Sans Mono", monospace ;
}
\stoptyping

The spacing before the first and after the last one differs from
the spacing between lines, so we need some extra directives:

\starttyping
verbatimlines+verbatimlines {
    display: block ;
    margin-top: 1em ;
}
\stoptyping

This will format code like the following with a bluish background
and inline verbatim with its complement:

\starttyping
<verbatimblock detail='typing'>
  <verbatimlines>
    <verbatimline>function sum(a,b)</verbatimline>
    <verbatimline>  print(a, b, a + b)</verbatimline>
    <verbatimline>end</verbatimline>
  </verbatimlines>
</verbatimblock>
\stoptyping

The hyperlinks need some attention. We need to make sure that only
the links and not the anchors get special formatting. After some
experimenting I arrived at this:

\starttyping
a[href] {
    text-decoration: none ;
    color: inherit ;
}

a[href]:hover {
    color: #770000 ;
    text-decoration: underline ;
}
\stoptyping

Tables are relatively easy to control. We have tabulate (nicer
for text) and natural tables (similar to the \HTML\ model). Both
get mapped into \HTML\ tables with \CSS\ directives. There is
some detail available so we see things like this:

\starttyping
tablecell[align=flushleft] {
    display: table-cell ;
    text-align: left ;
    padding: .1em ;
}
\stoptyping

It is not that hard to support more variants or detail in the
export but that will probably only happen when I find either a
good reason (a project), have some personal need, or when a user
asks for it. For instance images will need some special attention
(conversion etc.). Also, because we use \METAPOST\ all over the
place that needs special care as well, but a regular (novel like)
ebook will not have such resources.

\placefigure
  {A page from the the \HTML\ variant of the \CLD\ manual.}
  {\externalfigure[ebook-xhtml.png][width=\textwidth]}

As an extra a template file is generated that mentions all
elements used, like:

\starttyping
section[detail=summary] {
    display: block ;
}
\stoptyping

With the inline and display properties are already filled in. That
way I could see that I still had to add a couple of directives to
the final \CSS\ file. It also became clear that in the \CLD\
manual some math is used that gets tagged as \MATHML, so that
needs to be covered as well. \footnote {Some more advanced
\MATHML\ output will be available when the matrix related core
commands have been upgraded to \MKIV\ and extended to suit today's
needs.} Here we need to make some decisions as we export \UNICODE\
and need to consider support for less sophisticated fonts. On the
other hand, the w3c consortium has published \CSS\ for this
purpose so we can use these as starting point. It can be that
eventually some more tuning will be delegated to the \XHTML\
variant. This is not that much extra work as we have the (then
intermediate) \XML\ tree available. Thinking of it, we could
eventually end up with some kind of \CSS\ support in \CONTEXT\
itself.

It will take some experimenting and feedback from users to get the
export right, especially if we want to provide a convenient way to
produce so called \EPUB\ files directly. There is already some
support for this container format. If you have enabled \XHTML\
export, you can produce an \EPUB\ archive afterwards with:

\starttyping
mtxrun --script epub yourfile
\stoptyping

For testing the results, open source programs like \type {calibre}
are quite useful. It will probably take a while to figure out to
what extent we need to support formats like \EPUB, if only because
such formats are adapted on a regular basis.

\stopsection

\startsection[title=The future]

It is hard to predict the future. I can imagine that given the
user interface that evolved over ages paper books will not
disappear soon. Probably there will be a distinction between
read|-|once and throw|-|away books and those that you carry with
you your whole life as visible proof of that life. I can also
imagine that (if only for environmental reasons) ebooks (hopefully
with stable devices) will dominate. In that case traditional
bookshops will disappear and with them the need for publishers
that supply them. Self publishing will then be most interesting
for authors and maybe some of them (or their helpful friends) will
be charmed by \TEX\ and tinkering with the layout using the macro
language. I can also imagine that at some point new media (and I
don't consider an ebook a new medium) will dominate. And how about
science fiction becoming true: downloading stories and information
directly into our brains.

It reminds me of something I need to do some day soon: get rid of
old journals that I planned to read but never will. I would love
to keep them electronically but it is quite unlikely that they are
available and if so, it's unlikely that I want to pay for them
again. This is typically an area where I'd consider using an ebook
device, even if it's suboptimal. On the other hand, I don't
consider quitting my newspaper subscription yet as I don't see a
replacement for the regular coffeestop at the table where it sits
on and where we discuss the latest news.

The nice thing about an analogue camera is that the image carrier
has been standardized and you can buy batteries everywhere.
Compare this with their digital friends: all have different
batteries, there are all kind of memory cards, and only recently
some standardization in lenses shows up. There is a wide range of
resolutions and aspect ratios. Other examples of standardization
are nuts and bolts used in cars, although it took some time for
the metric system to catch on. Books have different dimensions but
it's not hard to deal with that property. Where desktop hardware
is rather uniform everything portable is different. For some
brands you need a special toolkit with every new device. Batteries
cannot be exchanged and there are quite some data carriers. On the
other hand, we're dealing with software and if we want we can
support data formats forever. The \MICROSOFT\ operating systems
have demonstrated that programs written years ago can still run on
updates. In addition \LINUX\ demonstrates that users can take and
keep control and create an independence from vendors. So, given
that we can still read document sources and given that they are
well structured, we can create history proof solutions. I don't
expect that the traditional publishers play a big role in this if
only because of their short term agendas and because changing
ownerships works against long term views. And programs like \TEX\
have already demonstrated to have a long live span, although it
must be said that in today's rapid upgrade cycles it takes some
courage to stay with it and its descendants. But downward
compatibility is high on the agenda of its users and user groups
which is good in the perspective of discussing stable ebooks.

Let's finish with an observation. Books often make a nice
(birthday) present and finding one that suits is part of the gift.
Currently a visible book has some advantages: when unwrapped it
can be looked at and passed around. It also can be a reason for
discussion and it has a visible personal touch. I'm not so sure if
vouchers for an ebook have the same properties. It probably feels
a bit like giving synthetic flowers. I don't know what percentage
of books is given as present but this aspect cannot be neglected.
Anyway, I wonder when I will buy my first ebook and for who.
Before that happens I've probably generated lots of them.

\stopsection

\stopcomponent
