		PLPLOT DOCUMENTATION IN THE DOCBOOK FORMAT
		    (Notes for contributors/developers)
     $Id: README.developers 12959 2014-01-26 02:50:33Z airwin $



Introduction
============

You will find in this directory tree the latest PLplot documentation in
DocBook 4.1 XML format.  You will find in the current file
(README.developers) instructions for building the documentation from the
DocBook source files in this directory. The documentation that is built
includes man pages of the API as well as complete versions of the
documentation in info, HTML, and PDF forms.

Configuring and Building
========================

If you would like to help with improving the content, you will want to build
the documentation from the DocBook source so you can immediately see and
evaluate the results of your efforts.

Here is how.  Simply add the cmake option -DBUILD_DOC=ON to your
normal cmake command-line options for the PLplot build.  The resulting
cmake step looks for all the tools required for the documentation
build and generates a WARNING message and turns off all or some
component of the documentation build if any of those required tools
are missing.  Currently, the tools searched for by our build system
are xmlto, dblatex, and xelatex.  Furthermore, at run-time xmlto and
dblatex (used below) automatically check for required tools, and
dblatex checks for missing fonts when generating the pdf results.
(Those required fonts are chosen by
doc/docbook/source/dblatex_stylesheet.xsl and are currently FreeSans,
FreeSerif, and FreeMono).

In sum, cmake WARNING messages with the -DBUILD_DOC=ON option _and_
run time messages from xmlto and dblatex should be sufficient for you
to figure out what DocBook/XML related tools and fonts you need to
install in order to build the documentation.  As far as we know
installation of the xmlto, dblatex, xelatex tools and the FreeSans,
FreeSerif, and FreeMono fonts should be sufficient, but if not, run-time
warnings should be generated.

N.B. As far as we know, the complete set of required tools is only
available on Linux so you will need a Linux system to do a
documentation build.

The DocBook Back-end Tool Chains
================================

PLplot uses a number of different applications to generate
PLplot documentation in various formats from our source DocBook XML files.

1.  man pages.

Our man pages are generated with a configured home-brew perl script
bin/api2man.pl(.in). That script uses XML::DOM::Parser to parse the
combination of plplotdoc.xml (for entity definitions) and api.xml
(the subset of our DocBook source files which describes our core
library's API) to obtain the information used to generate the man
pages.

2. info pages.

Our info pages are generated by a combination of 

bin/info-clean.pl --> db2x_xsltproc --> db2x_texixml --> makeinfo

where info-clean.pl is a home-brew perl script required (as far as I
can tell from reading comments in it) to get around some of the
db2x_xsltproc limitations.  db2x_xsltproc and db2x_texixml are from
the docbook2x package.

3. Our HTML results are generated with xmlto and configured at run
time with a CSS stylesheet, see doc/docbook/src/stylesheet.css.xsl.in.
The xmlto application is actually a convenience script that uses
XML/XSL to generate the HTML results as opposed to the previous
SGML/DSSSL approach used to create the HTML results for plplot-5.9.9
and previous.

4. Our PDF results are generated with "dblatex --backend=xetex" using
a combination of XML/XSL and xelatex to generate the PDF results (in
contrast to the SGML/DSSL tools used for that job for plplot-5.9.9 and
previous, and the XML/XSL "xmlto --with-fop" method used with
PLplot-5.9.10 to do that job.

5. Generation of dvi and PostScript forms of our documentation is
no longer available.

Validation
==========

Validation checks that the PLplot Docbook files in doc/docbook/src are
well-formed XML and correspond to the DocBook schema.  You can perform
such validation by typing the

make validate

command at the top of the build tree.  This make target is only available if
the PLplot CMake-based build system has found the onsgmls software
application (which is distributed as part of OpenSP) on your system. This
quick check works regardless of whether you decide to build the
documentation with -DBUILD_DOC=ON or not. Using "make validate" is
especially useful if you are just making a series of simple changes to the
files in doc/docbook/src, and you don't really feel it is necessary to check
every change by doing a complete documentation build.

In addition, the xmlto commands used to build the HTML and print
part of the documentation automatically include a validation step
with xmllint.  It turns out that xmllint is more sensitive than onsgmls
to DocBook XML errors.  On the other hand, onsgmls handles any
errors it finds with ease while xmllint tends to segfault when there
is a validation error.  So use "make validate"
first to detect any obvious validation errors using onsgmls to avoid
situations where xmlto's call to xmllint will segfault.

Testing the documentation that has been built.
==============================================

All tests are performed in the doc/dockbook/src subdirectory _of the
build tree_.  The given test commands are for the bash shell, and
$version is currently 5.9.10.  For the man pages, look carefully at
the style of the results.  For the rest, look carefully at the style
of the api chapter and the style/results near Table-3.4 in the
advanced chapter.  In particular look at how the following examples
render: the overline-underline example just prior to Table-3.4,
Table-3.4 of Greek symbols, Table-3.5 of the "Peace" word expressed in
various languages, and the mathematical symbols occurring in the
paragraph just after Table-3.5.

1. man pages.

(for MANPAGE in *.3plplot; do nroff -man $MANPAGE ; done) |less

Two of our man pages (plot3dcl.3plplot and plsurf3dl.3plplot)
currently contain utf8 symbols (for the math symbols ≤ and ≥).  Those
symbols do not render correctly using nroff -man (or any other
recommended nroff options such as -Tutf8).  However, they display
without issues if viewed with man (by copying the files to a temporary
directory man/man3 and viewing them with, e.g., "man -M./man
plot3dcl").

2. info pages.

info --file plplotdoc.info

3. web pages.

Browse with your favorite browser the index.html file that is generated
by xmlto.

4. pdf file

Browse with your favorite browser the plplot-$version.pdf file that
is generated by dblatex.

UTF-8 limitations of the current set of backend tools
=====================================================

The above set of DocBook backend tools has been chosen with the goal
of allowing essentially arbitrary UTF-8 strings into our DocBook
source.  And the advanced.xml part of our DocBook source constitutes a
simple test of how close we have come to that goal by including the
"Peace" word in all the human languages expressed in example 24 and by
also including some UTF-8 forms of math symbols.  Here are the current
results for how well those UTF-8 strings render for our various
backends.

Man pages.

These results are not affected since the man pages ignore the advanced
chapter, but (see discussion above) the api.xml file currently
contains two functions whose documentation uses UTF-8 math symbols,
and for those man pages the man command renders those symbols without
issues.  I assume what is going on here is the UTF-8 math symbols
are just being cleanly passed through to the man viewer ("less" in
this case) so with some exceptions (see comments on less below) good
rendering of arbitrary UTF-8 symbols will occur.

Info files.

These results are reasonably good.  All math glyphs and all but the
glyphs occurring in the Korean, Hindi, and Mandarin "Peace" words are
rendered correctly.  That's exactly the same set of missing glyphs
that occurs when I use "less" on advanced.xml so it is possible some
configuration adjustment for the non-GUI component of my system will
fix the missing glyphs that occur both for the info form of our
documentation and when using "less".

HTML results.

These are outstanding results with no issues that we are aware of.
All math glyphs and all the glyphs occurring in the set of "Peace"
words render without issues including the CTL languages like Hebrew,
Arabic, and Hindi.

PDF results.

These results are reasonably good.  All math glyphs and all but the
glyphs occurring in the Korean and Mandarin "Peace" words are present.
I attribute the missing glyph issues to missing Korean and Mandarin
glyphs in the chosen FreeSans, FreeSerif, and FreeMono fonts (more
about that choice below). The order of the glyphs in the Hindi Peace
word is not correct (last two glyphs switched) which is a common
complex text layout (CTL) issue when using unsophisticated software to
render Hindi.  However, the Peace word is laid out in the correct
(right-to-left) order for Arabic and Hebrew so there is some CTL
sophistication in the xelatex layout engine, and probably hope that
further layout issues such as the one for Hindi will be fixed.

More on the reason for the missing PDF glyphs.

It was a huge step forward in the tex world for the combination of
xelatex and the fontspec tex package to make a very large number of
UTF-8 glyphs potentially available in pdf results.  So I think the
above method of using dblatex with the xetex backend (which uses
xelatex and fontspec internally) is the best we can do to generate the
pdf form of our documentation as free as possible from unicode issues.
For example, the situation is much improved over the previous xmlto
method which severely limited the glyphs in the PDF form of our
results.

However, the fontspec package still has a fundamental limitation which
is you must use a specific rather than generic font for the sans,
serif, and mono cases. Specific fonts are never a good way to go for
documents containing different languages (such as the table of "Peace"
words I recently introduced into advanced.xml) since the user is
forced to compromise between font quality and comprehensiveness.  For
example, I have tried using the unifont font (which gives complete
coverage of glyphs for the basic multilingual plane), and the
resulting pdf form of our documentation has no missing glyphs.
However, that font is of extremely poor quality (scaled,
low-resolution bitmapped, monospaced fonts) which makes our
documentation look ugly so I am not going to use it.  Instead, I have
chosen (via doc/docbook/src/dblatex_stylesheet.xsl) to use FreeSans,
FreeSerif, and FreeMono fonts which have reasonable quality but which
have some missing glyphs (notably CJK glyphs that leave our Korean and
Mandarin "peace" words empty in the documentation).

The proper way to solve this issue (which would bring tex completely
into the modern unicode world) would be to modify fontspec so that
certain given font names are considered generic, (e.g., the
"sans-serif", "serif", and "mono-space" font names could be adopted
for this purpose to follow what is done in the SVG world).  For those
generic font names, the idea is the fontspec package would simply hand
off font selection to fontconfig which does a very nice job of
automatically selecting the ideal sans, serif, or mono system font to
provide a particular UTF-8 glyph.  I have asked the fontspec
maintainer about this potential feature several days before the
current writing, but so far he has not replied.  Therefore, it appears
at this time there will be no quick fixes to xelatex/fontspec with
regard to automatic selection of fonts for the generic case, and we
will be stuck for the foreseeable future with the specific font
approach.  That approach forces a compromise between complete glyph
coverage and reasonable quality.  Thus, as a result of avoiding truly
ugly fonts in our generated pdf documentation there are missing CJK
glyphs in the results.

Installing the generated documentation (and the rest of the generated website) at plplot.sf.net
===============================================================================================

Follow the directions in README.Release_Manager_Cookbook.  That file is
located in the top-level source tree.
