path: root/unicode/book.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
 "https://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [

<!ENTITY tr9ver  "42">
<!ENTITY tr14ver "45">
<!ENTITY tr15ver "50">
<!ENTITY tr24ver "31">
<!ENTITY tr29ver "37">
<!ENTITY tr51ver "18">

]>

<!--

Copyright 2014-2021 Double Precision, Inc.
See COPYING for distribution information.

-->

<article id="index">
  <title>Courier Unicode Library</title>

  <para>
    This library implements several algorithms related to the
    <ulink url="https://www.unicode.org/standard/standard.html">Unicode
    Standard</ulink>, featuring:
  </para>

  <itemizedlist>
    <listitem>
      <para>
	Both C and C++11 bindings, with a
	<link linkend="courier-unicode">complete manual page
	documentation set</link>.
      </para>
    </listitem>

    <listitem>
      <para>
	The library has all Unicode mappings compiled in as fast, compact,
	lookup table. The library does not need to load the Unicode database
	files at startup, every time.
      </para>
    </listitem>
    <listitem>
      <para>
	The library implements lookups uppercase, lowercase, and
	titlecase equivalents of a unicode character;
	<link linkend="unicode_grapheme_break">grapheme
	and word breaking</link> rules;
	<link linkend="unicode_line_break">line	breaking</link> rules;
	and the
	<link linkend="unicode_bidi">bi-directional
	algorithm</link>.
      </para>
    </listitem>
    <listitem>
      <para>
	The library implements <link linkend="unicode_canonical">canonical
	and compatibility decomposition and composition</link> of Unicode text;
	and the <link linkend="unicode_script">Unicode script property</link>.
      </para>
    </listitem>
    <listitem>
      <para>
	The library also implements ancillary functions, like looking up
	the unicode character that corresponds to some HTML 4.0
	entity (such as <quote>&amp;amp;</quote>, for example), and
	determining the normal width or a double-width status of a unicode
	character. Also, an adaptation of the
	<ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	<citerefentry><refentrytitle>iconv</refentrytitle>
	<manvolnum>3</manvolnum></citerefentry></ulink>
	API for this unicode library.
      </para>
    </listitem>
  </itemizedlist>

  <section id="status">
    <title>Current status</title>

    <para>
      The current release of the Courier Unicode library is based on the
      Unicode 13.0.0 standard.
    </para>
  </section>

  <section id="INSTALL">
    <title>Installation and usage</title>

    <para>
      Download the current version of the library from
      <ulink url="/download.html#unicode">https://www.courier-mta.org/download.html#unicode</ulink>.
      Use the downloaded tarball to prepare an appropriate installation
      package for your operating system distribution.
      The typical sequence of commands is:
    </para>

    <blockquote>
      <informalexample>
	<programlisting>
./configure    # Takes the default configure script options
make
make install DESTDIR=/tmp/courier-unicode-instimage # For example.</programlisting>
      </informalexample>
    </blockquote>

    <para>
      The library uses a stock configure script, <command>make</command>
      and <command>make install</command> command that respects the
      <varname>DESTDIR</varname> setting to create an installation image
      in the directory specified by <varname>DESTDIR</varname>.
    </para>

    <note>
      <para>
	<command>make install</command> does not take any explicit action
	to uninstall any older version of the library, or remove any files
	from an older version that do not exist any more in the new version.
	Use the created installation image to prepare an
	installable package in a native package format for your operating
	system distribution. Use your native system distribution's package
	manager to properly install and update this library.
      </para>
    </note>

    <para>
      To use the library, <quote>#include &lt;courier-unicode.h&gt;</quote> and link
      with <literal>-lcourier-unicode</literal>.
      The C++ compiler must have C++11 support. Minimum usable version of
      gcc appears to be gcc 4.4 with the <literal>-std=c++0x</literal> flag.
      Current versions of gcc use C++11, or higher, by default and do not
      require extra flags. For C++ code, as usual, the compiler and compilation
      flags for compiling any code that uses this library must be ABI-compatible
      too.
    </para>

    <para>
      The Courier Unicode library installs an
      <application>autoconf</application> macro to probe for C++11 support.
      In your <filename>configure.ac</filename>
    </para>

    <blockquote>
      <informalexample>
	<programlisting>
	  AX_COURIER_UNICODE_VERSION
	  AX_COURIER_UNICODE_CXXFLAGS

	  AC_SUBST(COURIER_UNICODE_CXXFLAGS)
	</programlisting>
      </informalexample>
    </blockquote>

    <para>
      Then, in <filename>Makefile.am</filename>:
    </para>

    <blockquote>
      <informalexample>
	<programlisting>
	  AM_CXXFLAGS = @COURIER_UNICODE_CXXFLAGS@
	</programlisting>
      </informalexample>
    </blockquote>

    <para>
      The <varname>AX_COURIER_UNICODE_VERSION</varname> macro
      checks the minimum library version, which defaults to the build
      version. An optional parameter explicitly specifies which version
      of the Courier Unicode library is the minimum version required, i.e.:
    </para>

    <blockquote>
      <informalexample>
	<programlisting>
	AX_COURIER_UNICODE_VERSION(2.2.0)</programlisting>
      </informalexample>
    </blockquote>

    <para>
      <varname>AX_COURIER_UNICODE_CXXFLAGS</varname> sets
      <varname>COURIER_UNICODE_CXXFLAGS</varname> to the appropriate option
      for older gcc compilers that require an option to enable C++11
      support.
    </para>

    <para>
      The starting point for the library documentation is
      <link linkend="courier-unicode">
	<citerefentry>
	  <refentrytitle>courier-unicode</refentrytitle>
	  <manvolnum>7</manvolnum></citerefentry></link>.
	  Refer to the included manual pages,
	  and
	  <ulink url="https://www.courier-mta.org/unicode/manpages.html"> the HTML
	  version of the man pages</ulink> for more information.
    </para>
  </section>

  <section id="manpages">
    <title>Manual pages</title>

    <section id="manpagesc">
      <title>C manual pages</title>
      <refentry id="courier-unicode">

	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>courier-unicode</refentrytitle>
	  <manvolnum>7</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>courier-unicode</refname>
	  <refpurpose>Courier Unicode Library</refpurpose>

	</refnamediv>

	<refsynopsisdiv>
	  <programlisting>
#include &lt;courier-unicode.h&gt;</programlisting>
	</refsynopsisdiv>
	<refsect1 id="courier_unicode_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    This library implements several algorithms related to the
	    <ulink url="https://www.unicode.org/standard/standard.html">Unicode
	    Standard</ulink>.
	    This library uses
	    <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html"
		   ><citerefentry><refentrytitle>iconv</refentrytitle>
	    <manvolnum>3</manvolnum></citerefentry></ulink> to convert
	      text in a given character set to unicode. Any character set
	      displayed by <command>iconv --list</command> can be specified
	      for the corresponding character set parameter. Additionally,
	      <filename>courier-unicode.h</filename> defines a special character
	      string <literal>unicode_x_imap_modutf7</literal> that specifies
	      the pseudo-character set for the modified-UTF7
	      encoding used in IMAP. This string can also be appended by
	      a space, and up to fifteen additional US-ASCII characters.
	      The resulting character set also encodes these additional
	      characters, in addition to unicode characters, with
	      modified-UTF7.
	  </para>

	  <para>
	    The C++ compiler must have C++11 support. Minimum usable version of
	    gcc appears to be gcc 4.4 with the <literal>-std=c++0x</literal>
	    flag. Current versions of gcc use C++11, or higher, by default and
	    do not require extra flags. Consult the packaging documentation
	    for the Courier Unicode Library for information on any
	    compiler flags that are needed to build software that links
	    with this library.
	  </para>
	</refsect1>
	<refsect1 id="courier_unicode_seealso">
	  <title>SEE ALSO</title>

	  <para>
	    <link linkend="unicode_bidi">
	      <citerefentry><refentrytitle>unicode_bidi</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_canonical">
	      <citerefentry><refentrytitle>unicode_canonical</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_category_lookup">
	      <citerefentry><refentrytitle>unicode_category_lookup</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry><refentrytitle>unicode_convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_default_chset">
	      <citerefentry><refentrytitle>unicode_default_chset</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_emoji_lookup">
	      <citerefentry><refentrytitle>unicode_emoji_lookup</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_html40ent_lookup">
	      <citerefentry><refentrytitle>unicode_html40ent_lookup</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_grapheme_break">
	      <citerefentry><refentrytitle>unicode_grapheme_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_line_break">
	      <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_script">
	      <citerefentry><refentrytitle>unicode_script</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_uc</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__bidi">
	      <citerefentry><refentrytitle>unicode::bidi</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__canonical">
	      <citerefentry><refentrytitle>unicode::canonical</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert">
	      <citerefentry><refentrytitle>unicode::iconvert::convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert_tocase">
	      <citerefentry><refentrytitle>unicode::iconvert::convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__fromu">
	      <citerefentry><refentrytitle>unicode::iconvert::fromu</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__tou">
	      <citerefentry><refentrytitle>unicode::iconvert::tou</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__tolower">
	      <citerefentry><refentrytitle>unicode::tolower</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__linebreak">
	      <citerefentry><refentrytitle>unicode::linebreak</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode__wordbreak">
	      <citerefentry><refentrytitle>unicode::wordbreak</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_bidi">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_bidi</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_bidi</refname>
	  <refname>unicode_bidi_calc_levels</refname>
	  <refname>unicode_bidi_calc_types</refname>
	  <refname>unicode_bidi_calc</refname>
	  <refname>unicode_bidi_reorder</refname>
	  <refname>unicode_bidi_cleanup</refname>
	  <refname>unicode_bidi_cleaned_size</refname>
	  <refname>unicode_bidi_logical_order</refname>
	  <refname>unicode_bidi_combinings</refname>
	  <refname>unicode_bidi_needs_embed</refname>
	  <refname>unicode_bidi_embed</refname>
	  <refname>unicode_bidi_embed_paragraph_level</refname>

	  <refname>unicode_bidi_direction</refname>
	  <refname>unicode_bidi_type</refname>
	  <refname>unicode_bidi_setbnl</refname>
	  <refname>unicode_bidi_mirror</refname>
	  <refname>unicode_bidi_bracket_type</refname>

	  <refpurpose>unicode bi-directional algorithm</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;&#10;&#10;unicode_bidi_level_t lr=UNICODE_BIDI_LR;</funcsynopsisinfo>

	    <funcprototype>
	      <funcdef>void <function>unicode_bidi_calc_types</function></funcdef>
              <paramdef>const char32_t *<parameter>p</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_type_t *<parameter>types</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>struct unicode_bidi_direction <function>unicode_bidi_calc_levels</function></funcdef>
              <paramdef>const char32_t *<parameter>p</parameter></paramdef>
              <paramdef>const unicode_bidi_type_t *<parameter>types</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t *<parameter>initial_embedding_level</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>struct unicode_bidi_direction <function>unicode_bidi_calc</function></funcdef>
              <paramdef>const char32_t *<parameter>p</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t *<parameter>initial_embedding_level</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_bidi_reorder</function></funcdef>
              <paramdef>char32_t *<parameter>string</parameter></paramdef>
              <paramdef>unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>void (*<parameter>reorder_callback</parameter>)(size_t, size_t, void *)</paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>size_t <function>unicode_bidi_cleanup</function></funcdef>
              <paramdef>char32_t *<parameter>string</parameter></paramdef>
              <paramdef>unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
	      <paramdef>int <parameter>options</parameter></paramdef>
              <paramdef>void (*<parameter>removed_callback</parameter>)(size_t, size_t, void *)</paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>size_t <function>unicode_bidi_cleaned_size</function></funcdef>
              <paramdef>const char32_t *<parameter>string</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
	      <paramdef>int <parameter>options</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_bidi_logical_order</function></funcdef>
              <paramdef>char32_t *<parameter>string</parameter></paramdef>
              <paramdef>unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
              <paramdef>void (*<parameter>reorder_callback</parameter>)(size_t index, size_t n, void *arg)</paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_bidi_combinings</function></funcdef>
              <paramdef>const char32_t *<parameter>string</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
	      <paramdef>void (*<parameter>combinings</parameter>)(unicode_bidi_level_t level, size_t level_start, size_t n_chars, size_t comb_start, size_t n_comb_chars, void *arg)</paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>
	    <funcprototype>
	      <funcdef>int <function>unicode_bidi_needs_embed</function></funcdef>
              <paramdef>const char32_t *<parameter>string</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t <parameter>*paragraph_embedding</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>size_t <function>unicode_bidi_embed</function></funcdef>
              <paramdef>const char32_t *<parameter>string</parameter></paramdef>
              <paramdef>const unicode_bidi_level_t *<parameter>levels</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
              <paramdef>void (*<parameter>emit</parameter>)(const char32_t *string, size_t n, int is_part_of_string, void *arg)</paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>char32_t <function>unicode_bidi_embed_paragraph_level</function></funcdef>
              <paramdef>const char32_t *<parameter>string</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
              <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>char32_t <function>bidi_mirror</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>char32_t <function>bidi_bracket_type</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
              <paramdef>unicode_bracket_type_t *<parameter>ret</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>struct unicode_bidi_direction <function>unicode_bidi_get_direction</function></funcdef>
              <paramdef>char32_t *<parameter>c</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>enum_bidi_type_t <function>unicode_bidi_type</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_bidi_setbnl</function></funcdef>
              <paramdef>char32_t *<parameter>p</parameter></paramdef>
              <paramdef>const unicode_bidi_type_t *<parameter>types</parameter></paramdef>
              <paramdef>size_t <parameter>n</parameter></paramdef>
	    </funcprototype>

	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_bidi_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions are related to the
	    <ulink url="https://www.unicode.org/reports/tr9/tr9-&tr9ver;.html"> Unicode Bi-Directional algorithm</ulink>.
	    They implement the algorithm up to and including step L2,
	    and provide additional functionality of returning miscellaneous
	    bi-directional-related metadata of Unicode characters. There's
	    also a basic algorithm that <quote>reverses</quote> the
	    bi-directional algorithm
	    and produces a Unicode string with bi-directional markers that
	    results in the same bi-directional string after reapplying the
	    algorithm.
	  </para>

	  <refsect2 id="unicode_bidi_calc_reorder">
	    <title>Calculating bi-directional rendering order</title>

	    <para>
	      The following process computes the rendering order of
	      characters according to the Unicode Bi-Directional algorithm:
	    </para>

	    <orderedlist>
	      <listitem>
		<para>
		  Allocate an array of
		  <structname>unicode_bidi_type_t</structname> that's the
		  same size as the Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Allocate an array of
		  <structname>unicode_bidi_level_t</structname> that's the
		  same size as the Unicode string.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  Use <function>unicode_bidi_calc_types</function>() to compute
		  the Unicode string's characters' bi-directional types,
		  and populate the
		  <structname>unicode_bidi_type_t</structname> buffer.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  Use <function>unicode_bidi_calc_levels</function>() to compute
		  the Unicode string's characters' bi-directional embedding
		  level (executes the Bi-Directional algorithm up to and
		  including step L1). This populates the
		  <structname>unicode_bidi_level_t</structname> buffer.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  Alternatively: allocate only the
		  <structname>unicode_bidi_level_t</structname> array
		  and use <function>unicode_bidi_calc</function>(), which
		  <function>malloc</function>()s the
		  <structname>unicode_bidi_type_t</structname> buffer,
		  calls <function>unicode_bidi_calc_levels</function>(),
		  and then <function>free</function>()s the buffer.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  Use <function>unicode_bidi_reorder</function>() to reverse
		  any characters in the string, according to the
		  algorithm (step L2), with an optional
		  callback that reports which ranges of characters get
		  reversed.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Use <function>unicode_bidi_cleanup</function>()
		  to remove the characters from the string which are used
		  by the bi-directional algorithm, and are not needed for
		  rendering the text.
		  <function>unicode_bidi_cleaned_size</function>() is
		  available to determine, in advance, how many characters
		  will remain.
		</para>
	      </listitem>
	    </orderedlist>

	    <para>
	      The parameters to
	      <function>unicode_bidi_calc_types</function>() are:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  A pointer to the Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Number of characters in the Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  A pointer to an array of
		  <structname>unicode_bidi_type_t</structname> values.
		  The caller is
		  responsible for allocating and deallocating this array,
		  which has the same size as the Unicode string.
		</para>
	      </listitem>
	    </itemizedlist>

	    <para>
	      The parameters to
	      <function>unicode_bidi_calc_levels</function>() are:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  A pointer to the Unicode string.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  A pointer to the buffer that was passed to
		  <function>unicode_bidi_calc_types</function>().
		</para>
	      </listitem>

	      <listitem>
		<para>
		  Number of characters in the Unicode string and the
		  <structname>unicode_bidi_type_t</structname> buffer.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  A pointer to an array of
		  <structname>unicode_bidi_level_t</structname> values.
		  The caller is
		  responsible for allocating and deallocating this array,
		  which has the same size as the Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  An optional pointer to a
		  <literal>UNICODE_BIDI_LR</literal> or
		  <literal>UNICODE_BIDI_RL</literal> value. This sets
		  the default paragraph direction level.
		  A null pointer computes the default paragraph direction
		  level based on the string, as specified by the "P" rules
		  of the bi-directional algorithm.
		</para>
	      </listitem>
	    </itemizedlist>

	    <para>
	      The parameters to <function>unicode_bidi_calc</function>() are
	      the same except for the
	      <structname>unicode_bidi_type_t</structname> pointer.
	      <function>unicode_bidi_calc</function>() allocates this
	      buffer by itself and calls
	      <function>unicode_bidi_calc_types</function>, and
	      destroys the buffer before returning.
	    </para>

	    <para>
	      <function>unicode_bidi_calc</function>()
	      and <function>unicode_bidi_calc_levels</function>() fill in the
	      <structname>unicode_bidi_level_t</structname> array with the
	      values corresponding to the embedding level of the
	      corresponding character,
	      according the Unicode Bidirection Algorithm (even values for
	      left-to-right ordering, and odd values for right-to-left
	      ordering).
	      A value of UNICODE_BIDI_SKIP designates directional markers
	      (from step X9).
	    </para>

	    <para>
	      <function>unicode_bidi_calc</function>()
	      and <function>unicode_bidi_calc_levels</function>()
	      return the resolved
	      paragraph direction level, which
	      always matches the passed in level, if specified, else it
	      reports the derived one. These functions return a
	      <structname>unicode_bidi_direction</structname> structure:
	    </para>

	    <informaltable border='0'>
	      <tgroup cols="3">
		<colspec colname='c1' />
		<colspec colname='c2' />
		<colspec colname='c3' />
		<tbody>
		  <row>
		    <entry namest='c1' nameend='c3'>struct&nbsp;<structname>unicode_bidi_direction</structname>&nbsp;{</entry>
		  </row>
		  <row>
		    <entry></entry>
		    <entry>unicode_bidi_level_t</entry>
		    <entry><varname>direction</varname>;</entry>
		  </row>
		  <row>
		    <entry></entry>
		    <entry>int</entry>
		    <entry><varname>is_explicit</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c1' nameend='c3'>};</entry>
		  </row>
		</tbody>
	      </tgroup>
	    </informaltable>
	    <para>
	      <varname>direction</varname> gives the paragraph embedding
	      level, <literal>UNICODE_BIDI_LR</literal> or
	      <literal> UNICODE_BIDI_RL</literal>.
	      <varname>is_explicit</varname> indicates whether:
	      the optional pointer to a
	      <literal>UNICODE_BIDI_LR</literal> or
	      <literal>UNICODE_BIDI_RL</literal> value was specified (and
	      returned in <varname>direction</varname>), or whether the
	      <varname>direction</varname> comes from an character with an
	      explicit direction indication.
	    </para>

	    <para>
	      <function>unicode_bidi_reorder</function>() takes the actual
	      unicode string together with the embedding values from
	      <function>unicode_bidi_calc</function> or
	      <function>unicode_bidi_calc_levels</function>(), then reverses the
	      bi-directional string, as specified by step L2 of the bi-directional
	      algorithm.
	      The parameters to
	      <function>unicode_bidi_reorder</function>() are:
	    </para>
	    <itemizedlist>
	      <listitem>
		<para>
		  A pointer to the Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  A pointer to an array of
		  <structname>unicode_bidi_level_t</structname> values.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Number of characters in the Unicode string and the
		  <structname>unicode_bidi_level_t</structname> array.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  An optional <varname>reorder_callback</varname> function
		  pointer.
		</para>
	      </listitem>
	    </itemizedlist>
	    <para>
	      A non-<literal>NULL</literal>
	      <parameter>reorder_callback</parameter> gets invoked to report
	      each reversed character range. The callback's first parameter
	      is the index of the first reversed character, the second parameter
	      is the number of reversed characters, starting at the given
	      index of the Unicode string.
	      The third parameter is the <parameter>arg</parameter> passthrough
	      parameter.
	    </para>

	    <para>
	      <function>unicode_bidi_reorder</function> modifies its
	      <parameter>string</parameter> and <parameter>levels</parameter>.
	      <parameter>reorder_callback</parameter> gets invoked after
	      reversing each consecutive range of values in the
	      <parameter>string</parameter> and <parameter>levels</parameter>
	      buffers. For example: <quote>reorder_callback(5, 7, arg)</quote>
	      reports that character indexes #5 through #11 got reversed.
	    </para>

	    <para>
	      A NULL <parameter>string</parameter> pointer leaves the
	      <parameter>levels</parameter> buffer unchanged, but still
	      invokes the <parameter>reorder_callback</parameter> as if
	      the character string, and their embedding values, were reversed.
	    </para>

	    <para>
	      The resulting string and embedding levels are in
	      <quote>rendering order</quote>, but still contain bi-directional
	      embedding, override, boundary-neutral, isolate, and marker
	      characters.
	      <function>unicode_bidi_cleanup</function>
	      removes these characters and directional markers.
	    </para>
	    <para>
	      The parameters to <function>unicode_bidi_cleanup</function>()
	      are:
            </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  The pointer to the unicode string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  A non-null pointer to the directional embedding level buffer,
		  of the same size as the string, also removes the corresponding
		  values from the buffer, and the remaining values in the
		  embedding level buffer get reset to
		  levels <literal>UNICODE_BIDI_LR</literal> and
		  <literal> UNICODE_BIDI_RL</literal>, only.
		</para>
              </listitem>

	      <listitem>
		<para>
		  The size of the unicode string and the directional embedding
		  buffer (if not NULL).
                </para>
              </listitem>

	      <listitem>
		<para>
		  A a bitmask that selects the following options
		  (or 0 if no options):
		</para>

		<variablelist>
		  <varlistentry>
		    <term><literal>UNICODE_BIDI_CLEANUP_EXTRA</literal></term>
		    <listitem>
		      <para>
			In addition to removing all embedding, override, and
			boundry-neutral characters as
			specified by step X9 of the bi-directional algorithm
			(the default behavior without this flag), also
			remove all isolation markers and implicit markers.
		      </para>
		    </listitem>
		  </varlistentry>

		  <varlistentry>
		    <term><literal>UNICODE_BIDI_CLEANUP_BNL</literal></term>
		    <listitem>
		      <para>
			Replace all characters classified as paragraph
			separators with a newline character.
		      </para>
		    </listitem>
		  </varlistentry>

		  <varlistentry>
		    <term><literal>UNICODE_BIDI_CLEANUP_CANONICAL</literal></term>
		    <listitem>
		      <para>
			A combined set of
			<literal>UNICODE_BIDI_CLEANUP_EXTRA</literal>
			and
			<literal>UNICODE_BIDI_CLEANUP_BNL</literal>,
		      </para>
		    </listitem>
		  </varlistentry>
		</variablelist>
	      </listitem>

	      <listitem>
		<para>
		  A pointer to a function that gets repeatedly invoked with the
		  index of the character that gets removed from the Unicode
		  string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  An opaque pointer that gets forwarded to the callback.
                </para>
              </listitem>
            </itemizedlist>
	    <para>
	      The function pointer (if not <literal>NULL</literal>)
	      gets invoked to report the index of each
	      removed character. The reported index is the index from the
	      original string, and the callback gets invoked in strict order,
	      from the first to
	      the last removed character (if any).
            </para>

	    <para>
	      The character string and the embedding level values resulting
	      from <function>unicode_bidi_cleanup</function>()
	      with the <literal>UNICODE_BIDI_CLEANUP_CANONICAL</literal>
	      are in
	      <quote>canonical rendering order</quote>.
	      <function>unicode_bidi_logical_order</function>(),
	      <function>unicode_bidi_needs_embed</function>() and
	      <function>unicode_bidi_embed</function>() require the
	      canonical rendering order for their string and embedding level
	      values.
            </para>
	    <para>
	      The parameters to <function>unicode_bidi_cleaned_size</function>()
	      are a pointer to the unicode string, its size, and
	      the bitmask option to <function>unicode_bidi_cleanup</function>().
            </para>
	  </refsect2>

	  <refsect2 id="unicode_bidi_embed">
	    <title>Embedding bi-directional markers in Unicode text strings</title>
            <para>
	      <function>unicode_bidi_logical_order</function>() rearranges
	      the string from rendering to its logical order.
	      <function>unicode_bidi_embed</function>() adds various
	      bi-directional markers to a Unicode string in canonical rendering
	      order. The resulting string is not guaranteed to be
	      identical to the
	      original Unicode bi-directional string. The algorithm is fairly
	      basic,
	      but the resulting bi-directional string produces the same
	      canonical rendering order after applying
	      <function>unicode_bidi_calc()</function> or
	      <function>unicode_bidi_calc_levels</function>(),
	      <function>unicode_reorder()</function> and
	      <function>unicode_bidi_cleanup()</function>
	      (with the canonical option),
	      with the same paragraph_embedding level.
	      <function>unicode_bidi_needs_embed</function>() attempts to
	      heuristically determine whether
	      <function>unicode_bidi_embed</function>() is required.
            </para>

	    <para>
	      <function>unicode_bidi_logical_order</function>() gets called
	      first, followed by
	      <function>unicode_bidi_embed</function>()
	      (or
	      <function>unicode_bidi_needs_embed</function>() in order to
	      determine whether bi-directional markers are required).
	      Finally, <function>unicode_bidi_embed_paragraph_level</function>()
	      optionally determines whether the resulting string's default
	      paragraph embedding level matches the one used for the actual
	      embedding direction, and if not returns a directional marker
	      to be prepended to the Unicode character string, as a hint.
            </para>
	    <para>
	      <function>unicode_bidi_logical_order</function>() factors in the
	      characters' embedding values, and the provided paragraph
	      embedding value
	      (<literal>UNICODE_BIDI_LR</literal> or
	      <literal>UNICODE_BIDI_RL</literal>), and rearranges the characters
	      and the embedding levels in left-to-right order, while
	      simultaneously
	      invoking the supplied reorder_callback indicating each range of
	      characters whose relative order gets reversed. The
	      <function>reorder_callback</function>() receives, as
	      parameters:
            </para>
	    <itemizedlist>
	      <listitem>
		<para>
		  The starting index of the first reversed character, in the
		  string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  Number of reversed characters.
                </para>
              </listitem>
	      <listitem>
		<para>
		  Forwarded <parameter>arg</parameter> pointer value.
                </para>
              </listitem>
            </itemizedlist>
	    <para>
	      This specifies a consecutive range of characters (and
	      directional  embedding values)
	      that get reversed (first character in the range becomes the
	      last character,
	      and the last character becomes the first character).
            </para>

	    <para>
	      After
	      <function>unicode_bidi_logical_order</function>(),
	      <function>unicode_bidi_embed</function>() progressively invokes
	      the passed-in callback with
	      the contents of a bi-directional unicode string.
	      The parameters to <function>unicode_bidi_embed</function>() are:
            </para>
            <itemizedlist>
	      <listitem>
		<para>
		  The Unicode string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The directional embedding buffer, in canonical
		  rendering order.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The size of the string and the embedding level buffer.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The paragraph embedding level, either
		  <literal>UNICODE_BIDI_LR</literal> or
		  <literal>UNICODE_BIDI_RL</literal>.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The pointer to the callback function.
                </para>
              </listitem>
	      <listitem>
		<para>
		  An opaque pointer argument that gets forwarded to the
		  callback function.
                </para>
              </listitem>
            </itemizedlist>
	    <para>
	      The callback receives pointers to
	      various parts of the original string that gets passed to
	      <function>unicode_bidi_embed</function>(), intermixed with
	      bi-directional markers,
	      overrides, and isolates. The callback's parameters are:
            </para>

            <itemizedlist>
	      <listitem>
		<para>
		  The pointer to a Unicode string.
                </para>
		<note>
		  <para>
		    It is not a given that the callback receives pointers
		    to progressively increasing pointers of the original
		    string that gets passed to
		    <function>unicode_bidi_embed</function>().
		    Some calls will be for individual bi-directional
		    markers, and
		    <function>unicode_bidi_embed</function>() also
		    performs some additional internal reordering, on the fly,
		    after <function>unicode_bidi_logical_order</function>()'s
		    big hammer.
                  </para>
                </note>
              </listitem>
	      <listitem>
		<para>
		  Number of characters in the Unicode string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  Indication whether the Unicode string pointer is pointing
		  to a part of the original Unicode string that's getting
		  embedded. Otherwise this must be some marker character that's
		  not present in the original Unicode string.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Forwarded <parameter>arg</parameter> pointer value.
                </para>
              </listitem>
            </itemizedlist>

	    <para>
	      The assembled unicode string should produce the same
	      canonical rendering order, for the same paragraph embedding
	      level.
	      <function>unicode_bidi_embed_paragraph_level</function>()
	      checks if the specified Unicode string computes the given
	      default paragraph embedding level and returns 0 if it matches.
	      Otherwise it returns a directional marker that should be
	      <emphasis>prepended</emphasis> to the Unicode string to allow
	      <function>unicode_bidi_calc</function>'s
	      (or <function>unicode_bidi_calc_levels</function>())
	      optional paragraph
	      embedding level pointer's value to be <literal>NULL</literal>,
	      but derive the same default embedding level.
	      The parameters to
	      <function>unicode_bidi_embed_paragraph_level</function>() are:
            </para>
            <itemizedlist>
	      <listitem>
		<para>
		  The Unicode string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The size of the string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The paragraph embedding level, either
		  <literal>UNICODE_BIDI_LR</literal> or
		  <literal>UNICODE_BIDI_RL</literal>.
                </para>
              </listitem>
	    </itemizedlist>

	    <para>
	      <function>unicode_bidi_needs_embed</function>() attempts to
	      heuristically determine whether the Unicode string, in logical
	      order, requires bi-directional markers.
	      The parameters to
	      <function>unicode_bidi_embed_paragraph_level</function>() are:
	    </para>
	    <itemizedlist>
	      <listitem>
		<para>
		  The Unicode string.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The directional embedding buffer, in logical order.
                </para>
              </listitem>
	      <listitem>
		<para>
		  The size of the string and the embedding level buffer.
                </para>
              </listitem>
	      <listitem>
		<para>
		  A pointer to an explicit paragraph embedding level, either
		  <literal>UNICODE_BIDI_LR</literal> or
		  <literal>UNICODE_BIDI_RL</literal>; or a
		  <literal>NULL</literal> pointer (see
		  <function>unicode_bidi_calc_types</function>()'s
		  explanation for this parameter).
                </para>
              </listitem>
	    </itemizedlist>

	    <para>
	      <function>unicode_bidi_needs_embed</function>() returns 0
	      if the Unicode string does not need explicit directional
	      markers, or 1 if it does. This is done by using
	      <function>unicode_bidi_calc()</function>,
	      <function>unicode_bidi_reorder()</function>,
	      <function>unicode_bidi_logical_order</function> and then
	      checking if the end result is different from what was passed
	      in.
	    </para>
          </refsect2>
	  <refsect2 id="unicode_bidi_combinings">
	    <title>Combining character ranges</title>

	    <para>
	      <function>unicode_bidi_combinings</function>() reports
	      consecutive sequences of one or more combining marks
	      in bidirectional text (which can be either in rendering or
	      logical order) that have the same embedding level. It takes
	      the following parameters:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>The Unicode string.</para>
	      </listitem>
	      <listitem>
		<para>
		  The directional embedding buffer, in logical
		  or rendering order. A <literal>NULL</literal> value for
		  this pointer is equivalent to a directional embedding
		  buffer with a level of 0 for every character in the Unicode
		  string.
                </para>
              </listitem>

	      <listitem>
		<para>
		  Number of characters in the Unicode string.
                </para>
              </listitem>

	      <listitem>
		<para>
		  The pointer to the callback function.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  An opaque pointer argument that gets forwarded to the
		  callback function.
                </para>
              </listitem>
            </itemizedlist>

	    <para>
	      The callback function gets invoked for every consecutive
	      sequence of one or more characters that have a canonical
	      combining class other than 0, and with the same
	      embedding level. The parameters to the callback function are:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>The embedding level of the combining characters.</para>
	      </listitem>
	      <listitem>
		<para>
		  The starting index of a consecutive sequence of all
		  characters with the same embedding level.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  The number of characters with the same embedding level.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  The starting index of a consecutive sequence of all
		  characters with the same embedding level
		  and a canonical combining
		  class other than 0. This will always be equal to or greater
		  than the value of the second parameter.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  The number of consecutive characters with the
		  characters with the same embedding level
		  and a canonical combining class other than 0.
		  The last character included in this sequence will always
		  be less than or equal to the last character in the sequence
		  defined by the second and the third parameters.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  The opaque pointer argument that was passed to
		  <function>unicode_bidi_combinings</function>.
                </para>
              </listitem>
	    </itemizedlist>

	    <para>
	      A consecutive sequence of Unicode characters with non-0 combining
	      classes but different embedding levels gets reported individually,
	      for each consecutive sequence with the same embedding level.
	    </para>

	    <para>
	      This function helps with reordering the combining characters in
	      right-to-left-rendered text.
	      Right-to-left text reversed by
	      <function>unicode_bidi_reorder</function>() results in combining
	      characters preceding their starter character. They get reversed
	      no differently than any other character.
	      The same thing also occurs after
	      <function>unicode_bidi_logical_order</function>() reverses
	      everything back.
	      Use <function>unicode_bidi_combinings</function> to identify
	      consecutive sequences of combining characters followed by their
	      original starter.
	    </para>

	    <para>
	      The callback may reorder the characters identified
	      by its third and the fourth parameters
	      in the manner described below.
	      <function>unicode_bidi_reorder</function>'s parameter is
	      pointers to a constant Unicode string; but it can modify the
	      string (via an out-of-band mutable pointer) subject to the
	      following conditions:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  The characters identified by the third and the fourth
		  parameter may be modified.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  If the last character in this sequence is not the last
		  character included in the range specified by the first
		  and the second character, then one more character after
		  the last character may also be modified.
		</para>

		<para>
		  This is, presumably, the original starter that preceded
		  the combining characters before the entire sequence was
		  reversed.
		</para>
	      </listitem>
	    </itemizedlist>

	    <para>
	      Here's an example of a callback that reverses
	      combining characters and their immediately-following starter
	      character:
	    </para>
	    <blockquote>
	      <informalexample>
		<programlisting><![CDATA[
void reorder_right_to_left_combining(unicode_bidi_level_t level,
                                     size_t level_start,
                                     size_t n_chars,
                                     size_t comb_start,
                                     size_t n_comb_chars,
                                     void *arg)
{
    /* Let's say that this is the Unicode string */
    char32_t *buf=(char32_t *)arg;

    if ((level & 1) == 0)
        return; /* Left-to-right text not reversed */

    char32_t *b=buf+comb_start;
    char32_t *e=b+n_comb_chars;

    /*
    ** Include the starter characters in the reversed range.
    ** The semantics of the combining characters with different
    ** embedding levels -- so they get reported here separately -- is
    ** not specified. This will reverse just the combining marks, and
    ** they're on their own.
    */

    if (comb_start + n_comb_chars < level_start + n_chars)
        ++e;

    while (b < e)
    {
        char32_t t;

        --e;
        t=*b;
        *b=*e;
        *e=t;
        ++b;
    }
}]]></programlisting>
	      </informalexample>
	    </blockquote>
	  </refsect2>
	  <refsect2 id="unicode_bidi_misc">
	    <title>Miscellaneous utility functions</title>

	    <para>
	      <function>unicode_bidi_get_direction</function>
	      takes a pointer to a unicode string, the number of
	      characters in the unicode string, and determines
	      default paragraph level level.
	      <function>unicode_bidi_get_direction</function> returns
	      a <literal>struct</literal> with the following fields:
	    </para>
	    <variablelist>
	      <varlistentry>
		<term><varname>direction</varname></term>
		<listitem>
		  <para>
		    This value is either <literal>UNICODE_BIDI_LR</literal>
		    or <literal>UNICODE_BIDI_RL</literal> (left to right or
		    right to left).
		  </para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><varname>is_explicit</varname></term>
		<listitem>
		  <para>
		    This value is a flag. A non-0 value indicates that
		    the embedding level was derived from an explicit
		    character type (<literal>L</literal>, <literal>R</literal>
		    or <literal>AL</literal>) from the stirng. A 0 value
		    indicates the default paragraph direction, no explicit
		    character was found in the string.
		  </para>
		</listitem>
	      </varlistentry>
	    </variablelist>
	    <para>
	      <function>unicode_bidi_type</function>
	      looks up each character's bi-directional character type.
	    </para>

	    <para>
	      <function>unicode_bidi_setbnl</function>
	      takes a pointer to a unicode string, a pointer to an
	      array of <classname>enum_bidi_type_t</classname> values and
	      the number of characters in the string and the array.
	      <function>unicode_bidi_setbnl</function> replaces all
	      paragraph separators in the unicode string with a newline
	      character (same as the <literal>UNICODE_BIDI_CLEANUP_BNL</literal>
	      option to <function>unicode_bidi_cleanup</function>.
	    </para>

	    <para>
	      <function>unicode_bidi_mirror</function>
	      returns the glyph that's a mirror image of the parameter
	      (i.e. an open parenthesis for a close parenthesis, and vice
	      versa); or the same value if there is no mirror image
	      (this is the <literal>Bidi_Mirrored=Yes</literal> property).
	    </para>

	    <para>
	      <function>unicode_bidi_bracket_type</function>
	      looks up each bracket character and returns its opposite, or
	      the same value if the character is not a bracket that has an
	      opposing bracket character
	      (this is the <literal>Bidi_Paired_Bracket_type</literal>
	      property).
	      A non-NULL <parameter>ret</parameter> gets initialized to
	      either <literal>UNICODE_BIDI_o</literal>,
	      <literal>UNICODE_BIDI_c</literal> or
	      <literal>UNICODE_BIDI_n</literal>.
	    </para>
	  </refsect2>
	</refsect1>
	<refsect1 id="courier_unicode_bidi_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <ulink url="https://www.unicode.org/reports/tr9/tr9-&tr9ver;.html">TR-9</ulink>,
	    <link linkend="unicode__bidi">
	      <citerefentry><refentrytitle>unicode::bidi</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_canonical">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_canonical</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_canonical</refname>
	  <refname>unicode_ccc</refname>
	  <refname>unicode_decomposition_init</refname>
	  <refname>unicode_decomposition_deinit</refname>
	  <refname>unicode_decompose</refname>
	  <refname>unicode_decompose_reallocate_size</refname>
	  <refname>unicode_compose</refname>
	  <refname>unicode_composition_init</refname>
	  <refname>unicode_composition_deinit</refname>
	  <refname>unicode_composition_apply</refname>

	  <refpurpose>unicode canonical normalization and denormalization</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
	      <funcdef>unicode_canonical_t <function>unicode_canonical</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>uint8_t <function>unicode_ccc</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_decomposition_init</function></funcdef>
	      <paramdef>unicode_decomposition_t *<parameter>info</parameter></paramdef>
	      <paramdef>char32_t *<parameter>string</parameter></paramdef>
	      <paramdef>size_t *<parameter>string_size</parameter></paramdef>
	      <paramdef>void *<parameter>arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>int <function>unicode_decompose</function></funcdef>
	      <paramdef>unicode_decomposition_t *<parameter>info</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_decomposition_deinit</function></funcdef>
	      <paramdef>unicode_decomposition_t *<parameter>info</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>size_t <function>unicode_decompose_reallocate_size</function></funcdef>
	      <paramdef>unicode_decomposition_t *<parameter>info</parameter></paramdef>
	      <paramdef>const size_t *<parameter>sizes</parameter></paramdef>
	      <paramdef>size_t <parameter>n</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>int <function>unicode_compose</function></funcdef>
	      <paramdef>char32_t *<parameter>string</parameter></paramdef>
	      <paramdef>size_t <parameter>string_size</parameter></paramdef>
	      <paramdef>int <parameter>flags</parameter></paramdef>
	      <paramdef>size_t *<parameter>new_size</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>int <funcname>unicode_composition_init</funcname></funcdef>
	      <paramdef>const char32_t *<parameter>string</parameter></paramdef>
	      <paramdef>size_t <parameter>string_size</parameter></paramdef>
	      <paramdef>int <parameter>flags</parameter></paramdef>
	      <paramdef>unicode_composition_t *<parameter>compositions</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <function>unicode_composition_deinit</function></funcdef>
	      <paramdef>unicode_composition_t *<parameter>compositions</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>size_t <function>unicode_composition_apply</function></funcdef>
	      <paramdef>char32_t *<parameter>string</parameter></paramdef>
	      <paramdef>size_t <parameter>string_size</parameter></paramdef>
	      <paramdef>unicode_composition_t *<parameter>compositions</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_canonical_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions compose or decompose a Unicode string into a
	    canonical or a compatible normalized form.
	  </para>

	  <para>
	    <function>unicode_canonical</function>() looks up the
	    character's
	    <ulink url="https://www.unicode.org/reports/tr15/tr15-&tr15ver;.html
">canonical
	    and compatibility mapping</ulink>.

	    <function>unicode_canonical</function>() returns a structure
	    with the following fields:
	  </para>

	  <variablelist>
	    <varlistentry>
	      <term><structfield>canonical_chars</structfield></term>
	      <listitem>
		<para>
		  A pointer to the canonical or equivalent representation
		  of the character.
	        </para>
	      </listitem>
	    </varlistentry>
	    <varlistentry>
	      <term><structfield>n_canonical_chars</structfield></term>
	      <listitem>
		<para>
		  Number of characters in the
		  <structfield>canonical_chars</structfield>.
	        </para>
	      </listitem>
	    </varlistentry>
	    <varlistentry>
	      <term><structfield>format</structfield></term>
	      <listitem>
		<para>
		  A value of <literal>UNICODE_CANONICAL_FMT_NONE</literal>
		  indicates a canonical mapping, other values indicate
		  a compatibility equivalent mapping.
	        </para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <para>
	    A NULL <structfield>canonical_chars</structfield> (with a 0
	    <structfield>n_canonical_chars</structfield>) indicates
	    that the character has no canonical or compatibility
	    equivalence.
	  </para>

	  <para>
	    <function>unicode_ccc</function>() returns the character's
	    canonical combining class value.
	  </para>

	  <para>
	    <function>unicode_decomposition_init</function>(),
	    <function>unicode_decompose</function>()
	    and <function>unicode_decomposition_deinit</function>()
	    implement a complete interface for decomposing a
	    Unicode string:
	  </para>

	  <blockquote>
	    <informalexample>
	      <programlisting><![CDATA[
unicode_decomposition_t info;

unicode_decomposition_init(&info, before, (size_t)-1, NULL);
info.decompose_flags=UNICODE_DECOMPOSE_FLAG_QC;
unicode_decompose(&info);
unicode_decomposition_deinit(&info);]]></programlisting>
	    </informalexample>
	  </blockquote>

	  <para>
	    <function>unicode_decomposition_init</function>() initializes
	    a new <classname>unicode_decomposition_t</classname> structure,
	    that gets passed in as its first parameter.
	    The second parameter is a pointer to a Unicode string,
	    with the number of characters in the string in the third parameter.
	    A string size
	    of <literal>-1</literal> indicates a
	    <literal>\0</literal>-terminated string and calculates its
	    <varname>string_size</varname> (which does not include the
	    trailing <literal>\0</literal>.
	    The last parameter is a <literal>void *</literal>, an opaque
	    pointer that gets stored in the initialized
	    <classname>unicode_decomposition_t</classname> object:
	  </para>
	  <blockquote>
	    <informaltable border='0' colsep='0'>
	      <tgroup cols="3">
		<colspec colname='c1' />
		<colspec colname='c2' />
		<colspec colname='c3' />
		<colspec colname='c4' />
		<colspec colname='c5' />
		<tbody>
		  <row>
		    <entry namest='c1' nameend='c5'>typedef struct&nbsp;<structname>unicode_decomposition</structname>&nbsp;{</entry>
		  </row>
		  <row>
		    <entry namest='c2'>char32_t</entry>
		    <entry namest='c3' nameend='c5'>*<varname>string</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>size_t</entry>
		    <entry namest='c3' nameend='c5'><varname>string_size</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>int</entry>
		    <entry namest='c3' nameend='c5'><varname>decompose_flags</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>int</entry>
		    <entry namest='c3' nameend='c5'>(*<varname>reallocate)(</varname></entry>
		  </row>
		  <row>
		    <entry namest='c4'>struct&#160;unicode_decomposition</entry>
		    <entry>*<varname>info</varname>,</entry>
		  </row>
		  <row>
		    <entry namest='c4'>const&#160;size_t</entry>
		    <entry>*<varname>offsets</varname>,</entry>
		  </row>
		  <row>
		    <entry namest='c4'>const&#160;size_t</entry>
		    <entry>*<varname>sizes</varname>,</entry>
		  </row>
		  <row>
		    <entry namest='c4'>size_t</entry>
		    <entry><varname>n</varname></entry>
		  </row>
		  <row>
		    <entry namest='c3' align='right'>);</entry>
		  </row>
		  <row>
		    <entry namest='c2'>void</entry>
		    <entry namest='c3' nameend='c5'>*<varname>arg</varname>;</entry>
		    </row>q
		    <row>
		      <entry namest='c1' nameend='c5'>} unicode_decomposition_t;</entry>
		    </row>
		</tbody>
	      </tgroup>
	    </informaltable>
	  </blockquote>

	  <para>
	    <function>unicode_decompose</function>() proceeds and decomposes
	    the <varname>string</varname> and replaces it with its
	    decomposed <varname>string</varname> version.
	  </para>

	  <para>
	    <classname>unicode_decomposition_t</classname>'s
	    <varname>string</varname>,
	    <varname>string_size</varname> and
	    <varname>arg</varname> are copies of
	    <function>unicode_decomposition_init</function>'s parameters.
	    <function>unicode_decomposition_init</function>
	    initializes all other fields to their default values.
	  </para>

	  <para>
	    The <varname>decompose_flags</varname> bitmask gets initialized to
	    0, and is a bit mask:
	  </para>

	  <variablelist>
	    <varlistentry>
	      <term><literal>UNICODE_DECOMPOSE_FLAG_QC</literal></term>
	      <listitem>
		<para>
		  Check each character's appropriate
		  <quote>quick check</quote> property
		  and skip decomposing Unicode characters that would
		  get re-composed by
		  <function>unicode_composition_apply</function>().
		</para>
	      </listitem>
	    </varlistentry>
	    <varlistentry>
	      <term><literal>UNICODE_DECOMPOSE_FLAG_COMPAT</literal></term>
	      <listitem>
		<para>
		  Perform a compatibility decomposition instead of a
		  canonical decomposition.
		</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <para>
	    <varname>reallocate</varname> is a pointer to a function that
	    gets called to reallocate a larger <varname>string</varname>.
	    <function>unicode_decompose</function>() determines which characters
	    in the <varname>string</varname> need decomposing and calls
	    the <varname>reallocate</varname> function pointer
	    zero or more times.
	    Each call to <varname>reallocate</varname> passes information
	    about where new characters will get inserted into the
	    <varname>string</varname>.
	  </para>

	  <para>
	    <varname>reallocate</varname> only needs to grow the size of the
	    buffer where
	    <varname>string</varname> points so that it's big enough to hold
	    a larger, decomposed string; then update
	    <varname>string</varname> accordingly.
	    <varname>reallocate</varname> should not update
	    <varname>string_size</varname> or make any changes to the existing
	    <varname>string</varname>, that's
	    <function>unicode_decompose</function>()'s job
	    (after <varname>reallocate</varname> returns).
	  </para>

	  <para>
	    The <varname>reallocate</varname> callback function receives
	    the following parameters.
	  </para>

	  <itemizedlist>
	    <listitem>
	      <para>
		A pointer to the
		<classname>unicode_decomposition_t</classname> and, notably,
		its <varname>arg</varname>.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		A pointer to the array of offset indexes in the
		<varname>string</varname> where new characters will get
		inserted in order to hold the decomposed string.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		A pointer to the array that holds the number of characters
		that get inserted each corresponding offset.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		The size of the two arrays.
	      </para>
	    </listitem>
	  </itemizedlist>
	  <para>
	    <varname>reallocate</varname> must update the
	    <varname>string</varname> if necessary to hold at least
	    the number of characters that's the sum total of the
	    initial <varname>string_size</varname> and the sum total of al
	    <varname>sizes</varname>.
	  </para>

	  <para>
	    <function>unicode_decomposition_init</function>() initializes
	    the <varname>reallocate</varname> pointer to a default
	    implementation that uses
	    <citerefentry>
	      <refentrytitle>realloc</refentrytitle>
	      <manvolnum>3</manvolnum>
	    </citerefentry>
	    and updates <varname>string</varname> with its return value.
	    The application can use its own
	    <varname>reallocate</varname> to handle this task on its own,
	    and use <function>unicode_decompose_reallocate_size</function>
	    to compute the minimum string size:
	  </para>

	  <blockquote>
	    <informalexample>
	      <programlisting><![CDATA[
size_t unicode_decompose_reallocate_size(unicode_decomposition_t *info,
                                         const size_t *sizes,
                                         size_t n)
{
    size_t i;
    size_t new_size=info->string_size;

    for (i=0; i<n; ++i)
        new_size += sizes[i];

    return new_size;
}]]>
	      </programlisting>
	    </informalexample>
	  </blockquote>

	  <para>
	    The <varname>reallocate</varname> function
	    returns 0 on success and
	    a non-0 error code to report a failure; and
	    <varname>unicode_decompose</varname>() does the same.
	    The only error condition from
	    <varname>unicode_decompose</varname>() is a non-0 error code
	    from the <varname>reallocate</varname> function. Otherwise:
	    a successful decomposition results in
	    <varname>unicode_decompose</varname>() returning 0 and
	    <function>unicode_decomposition_init</function>()'s
	    <varname>string</varname> pointing to the decomposed string
	    and <varname>string_size</varname> giving the number of
	    characters in the decomposed string.
	  </para>

	  <note>
	    <para>
	      <varname>string_size</varname> does not include the trailing
	      <literal>\0</literal> character.
	      The input string also has its
	      <varname>string_size</varname> specified without counting its
	      <literal>\0</literal> character.
	      The default implementation of <varname>reallocate</varname>
	      allocates an extra <classname>char32_t</classname> ands sets it
	      to a <literal>\0</literal>. Therefore:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  If the Unicode string before decomposition has a trailing
		  <literal>\0</literal> and no decomposition occurs, and
		  no calls to <varname>reallocate</varname> takes place:
		  the <varname>string</varname> in the
		  <classname>unicode_decomposition_t</classname> is unchanged
		  and it's still
		  <literal>\0</literal>-terminated.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  The default <varname>reallocate</varname> allocates an
		  extra <classname>char32_t</classname> ands sets it
		  to a <literal>\0</literal>; and it takes care of
		  that for the decomposed string.
		</para>
	      </listitem>

	      <listitem>
		<para>
		  An application that provides its own replacement
		  <varname>reallocate</varname> is responsible for doing
		  the same, if it wants the decomposed string to be
		  <literal>\0</literal> terminated.
		</para>
	      </listitem>
	    </itemizedlist>
	  </note>

	  <note>
	    <para>
	      Multiple calls to the <varname>reallocate</varname> callback
	      are possible. Each call to <varname>reallocate</varname>
	      reflect the prior calls' decompositions. Example:
	      the original string has five characters and the first call
	      to <varname>reallocate</varname> had two offsets, at position
	      1 and 3, with a value of 1 for their both
	      <varname>sizes</varname>.
	      This effects transforming an original Unicode string
	      "AAAAA" into
	      "AXAAXAA" (with <quote>A</quote> representing unspecified
	      characters in the original string, and <quote>X</quote> showing
	      the two characters added in the first call to
	      <function>reallocate</function>.
	    </para>

	    <para>
	      A second call to <varname>varname</varname> with am offset
	      at position 4, and a size of 1, results in the updated
	      string of "AXAAYXAA" (with <quote>Y</quote>) marking an
	      unspecified character inserted by the second call.
	    </para>
	  </note>

	  <note>
	    <para>
	      Unicode string decomposition involves replacing a given
	      Unicode character with one or more other characters.
	      The sizes given to <varname>reallocate</varname> reflect the
	      net addition to the Unicode string. For example: decomposing
	      one Unicode character into three decomposed characters results
	      in a call to <varname>reallocate</varname> reporting an
	      insert of two more characters.
	    </para>
	  </note>

	  <note>
	    <para>
	      <varname>offsets</varname> actually report the indices
	      of each Unicode character that's getting decomposed. A 1:1
	      decomposition of a Unicode Character gets reported as an
	      additional <varname>sizes</varname> entry of 0.
	    </para>
	  </note>

	  <para>
	    <function>unicode_decomposition_deinit</function>() releases
	    all resources and destroys the
	    <classname>unicode_decomposition_t</classname>; it is no longer
	    valid.
	  </para>

	  <note>
	    <para>
	      <function>unicode_decomposition_deinit</function>() does not
	      <citerefentry>
		<refentrytitle>free</refentrytitle>
		<manvolnum>3</manvolnum>
	      </citerefentry>
	      the <varname>string</varname>. The original string gets passed
	      in to <function>unicode_decomposition_init</function>() and
	      the decomposed string is left in the <varname>string</varname>.
	    </para>
	  </note>
	  <para>
	      The default implementation of the
	      <varname>reallocate</varname> function assumes the
	      <varname>string</varname> is a
	      <citerefentry>
		<refentrytitle>malloc</refentrytitle>
		<manvolnum>3</manvolnum>
		</citerefentry>-ed string, and
		<function>realloc</function>s it.
	  </para>

	  <note>
	    <para>
	      At this time
	      <function>unicode_decomposition_deinit</function>() does
	      nothing. All code should explicitly call it in order to
	      remain forward-compatible (at the source level).
	    </para>
	  </note>

	  <para>
	    <function>unicode_compose</function>() performs a canonical
	    composition of a decomposed string. Its parameters are:
	  </para>

	  <itemizedlist>
	    <listitem>
	      <para>
		A pointer to the decomposed Unicode string.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		The number of characters in the Unicode string.
		The Unicode string does not need to be
		<literal>\0</literal>-terminated; if it is this number
		does not include it.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		A flags bitmask, which can have the following values:
	      </para>

	      <variablelist>
		<varlistentry>
		  <term><literal>UNICODE_COMPOSE_FLAG_REMOVEUNUSED</literal></term>
		  <listitem>
		    <para>
		      Remove all combining marks after doing all canonical
		      compositions. Normally any unused combining marks
		      are left in place, in the combined text. This option
		      removes them.
		    </para>
		  </listitem>
		</varlistentry>
		<varlistentry>
		  <term><literal>UNICODE_COMPOSE_FLAG_ONESHOT</literal></term>
		  <listitem>
		    <para>
		      Perform canonical composition once per character, and
		      do not attempt to combine any resulting combined
		      characters again.
		    </para>
		  </listitem>
		</varlistentry>
	      </variablelist>
	    </listitem>

	    <listitem>
	      <para>
		A non-<literal>NULL</literal> pointer to a
		<classname>size_t</classname>.
	      </para>

	      <para>
		A successful composition sets this <classname>size_t</classname>
		to the number of characters in the combined string, and returns
		0. The combined string gets
		placed back into the <parameter>string</parameter> parameter,
		this string gets combined in place and this gives the
		size of the combined string.
	      </para>

	      <para>
		<function>unicode_compose</function>() returns a non-zero
		value to indicate an error.
	      </para>
	    </listitem>
	  </itemizedlist>

	  <para>
	    <function>unicode_composition_init</function>(),
	    <function>unicode_composition_apply</function>()
	    and <function>unicode_composition_deinit</function>()
	    implement a detailed interface for canonical composition
	    of a decomposed Unicode string:
	  </para>

	  <blockquote>
	    <informalexample>
	      <programlisting><![CDATA[
unicode_compositions_t compositions;

if (unicode_composition_init(str, strsize, flags, &compositions) == 0)
{
    size_t new_size=unicode_composition_apply(str, strsize, &compositions);

    unicode_composition_deinit(&compositions);
}]]></programlisting>
	    </informalexample>
	  </blockquote>

	  <para>
	    The first two parameters to both
	    <function>unicode_composition_init</function>()
	    and
	    <function>unicode_composition_apply</function>()
	    are the same:
	    the Unicode string and the number of characters (not including
	    any trailing <literal>\0</literal> character) in the Unicode string.
	  </para>

	  <para>
	    <function>unicode_composition_init</function>()'s additional
	    parameters are: any optional flags
	    (see <function>unicode_compose()</function> for a list of
	    available flags), and the address of a
	    <classname>unicode_composition_t</classname> object.
	    A non-0 return from
	    <function>unicode_composition_init</function>() indicates an
	    error.
	    <function>unicode_composition_init</function>() indicates success
	    by returning 0 and initializing the
	    <classname>unicode_composition_t</classname>'s object
	    which contains a pointer to an array of pointers to
	    of <structname>unicode_compose_info</structname> objects, and
	    the number of pointers.
	    <function>unicode_composition_init</function>() does not change
	    the string; the only thing it does is initialize the
	    <structname>unicode_composition_t</structname> object.
	  </para>

	  <para>
	    <function>unicode_composition_apply</function>() applies the
	    compositions to the <varname>string</varname>, in place, and
	    returns the new size of the <varname>string</varname>
	    (also not including the <literal>\0</literal> byte, however it
	    does append one if the composed string is smaller, so the
	    composed string is <literal>\0</literal>-terminated if the
	    decomposed string was).
	  </para>

	  <para>
	    It is necessary to call
	    <function>unicode_composition_deinit</function>() to free all
	    memory that was allocated for the
	    <classname>unicode_composition_t</classname> object:
	  </para>

	  <blockquote>




	    <informaltable border='0' colsep='0'>
	      <tgroup cols="3">
		<colspec colname='c1' />
		<colspec colname='c2' />
		<colspec colname='c3' />
		<tbody>
		  <row>
		    <entry namest='c1' nameend='c3'>struct&nbsp;<structname>unicode_compose_info</structname> {</entry>
		  </row>

		  <row>
		    <entry namest='c2'>size_t</entry>
		    <entry><varname>index</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>size_t</entry>
		    <entry><varname>n_composed</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>char32_t</entry>
		    <entry>*<varname>composition</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>size_t</entry>
		    <entry><varname>n_composition</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c1' nameend='c3'>};</entry>
		  </row>
		  <row>
		    <entry namest='c1' nameend='c3'>&nbsp;</entry>
		  </row>
		  <row>
		    <entry namest='c1' nameend='c3'>typedef&nbsp;struct&nbsp;{</entry>
		  </row>

		  <row>
		    <entry namest='c2'>struct&nbsp;<structname>unicode_compose_info</structname></entry>
		    <entry>**<varname>compositions</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c2'>size_t</entry>
		    <entry><varname>n_compositions</varname>;</entry>
		  </row>
		  <row>
		    <entry namest='c1' nameend='c3'>} unicode_composition_t;</entry>
		  </row>
		</tbody>
	      </tgroup>
	    </informaltable>
	  </blockquote>

	  <para>
	    <varname>index</varname> gives the character index in the
	    <varname>string</varname> where each composition occurs.
	    <varname>n_composed</varname> gives the number of characters
	    in the original string that get composed.
	    The composed characters are the
	    <varname>composition</varname>;
	    and <varname>n_composition</varname> gives the
	    number of composed characters.
	  </para>

	  <para>
	    Effectively: at the <varname>index</varname> position in the
	    original string, #<varname>n_composed</varname> characters get
	    removed and there are #<varname>n_composition</varname>
	    characters that replace them (always <varname>n_composed</varname>
	    or less).
	  </para>

	  <note>
	    <para>
	      The <literal>UNICODE_COMPOSE_FLAG_REMOVEUNUSED</literal> flag
	      has the effect of including
	      the combining marks that did not get combined
	      in the <varname>n_composed</varname> count. It's possible that,
	      in this case, <varname>n_composition</varname> is 0.
	      This indicates complete removal of the combining marks, without
	      anything getting combined in their place.
	    </para>
	  </note>

	  <para>
	    <function>unicode_composition_init</function>()
	    sets <classname>unicode_composition_t</classname>'s
	    <varname>compositions</varname> pointer to an array of
	    pointers to <structname>unicode_compose_info</structname>s
	    that are sorted according to their <varname>index</varname>.
	    <varname>n_compositions</varname> gives the number of pointers
	    in the array, and is 0 if there are no compositions, the
	    array is empty.
	    The empty array gets interpreted accordingly
	    when it gets passed to
	    <function>unicode_composition_apply</function>() and
	    <function>unicode_composition_deinit</function>(): nothing
	    happens. <function>unicode_composition_apply</function>()
	    simply returns the size of the unchanged <varname>string</varname>,
	    and <function>unicode_composition_deinit</function>() does a
	    pro-forma cleanup.
	  </para>
	</refsect1>
	<refsect1 id="unicode_canonical_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <ulink url="https://www.unicode.org/reports/tr15/tr15-&tr15ver;.html">TR-15</ulink>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
		<link linkend="unicode__canonical">
		  <citerefentry>
		  <refentrytitle>unicode::canonical</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_category_lookup">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_category_lookup</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_category_lookup</refname>
	  <refname>unicode_isalnum</refname>
	  <refname>unicode_isalpha</refname>
	  <refname>unicode_isblank</refname>
	  <refname>unicode_isdigit</refname>
	  <refname>unicode_isgraph</refname>
	  <refname>unicode_islower</refname>
	  <refname>unicode_ispunct</refname>
	  <refname>unicode_isspace</refname>
	  <refname>unicode_isupper</refname>

	  <refpurpose>unicode character categorization</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
	      <funcdef>uint32_t <function>unicode_category_lookup</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isalnum</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isalpha</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isblank</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isdigit</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isgraph</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_islower</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_ispunct</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isspace</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_isupper</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_category_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    <function>unicode_category_lookup</function>() looks up the
	    <ulink url="https://unicode.org/notes/tn36/">unicode character's
	    categorization</ulink>.
	    <function>unicode_category_lookup</function>() returns a 32 bit
	    value.
	    The value's
	    <symbol>UNICODE_CATEGORY_1</symbol> bits specify the first level
	    of the unicode character's category, with
	    <symbol>UNICODE_CATEGORY_2</symbol>,
	    <symbol>UNICODE_CATEGORY_3</symbol>, and
	    <symbol>UNICODE_CATEGORY_4</symbol> bits specifying the 2nd,
	    3rd, and 4th level, if given. A value of 0 for each corresponding
	    bit set indicates that no category is specified for this level,
	    for this character; otherwise the possible values are defined
	    in <filename>&lt;courier-unicode.h&gt;</filename>.
	  </para>

	  <para>
	    The remaining functions implement comparable equivalents of
	    their non-unicode versions in the standard C library, as follows:
	  </para>

	  <variablelist>
	    <varlistentry>
              <term><function>unicode_isalnum</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <function>unicode_isalpha</function>() or
		  <function>unicode_isdigit</function>().
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isalpha</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <symbol>UNICODE_CATEGORY_1_LETTER</symbol>.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isblank</function>()</term>
	      <listitem>
		<para>
		  Return non-0 for
		  <symbol>TAB</symbol>, and all
		  <symbol>UNICODE_CATEGORY_2_SPACE</symbol>.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isdigit</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <symbol>UNICODE_CATEGORY_1_NUMBER</symbol>
		  | <symbol>UNICODE_CATEGORY_2_DIGIT</symbol>,
		  only (no third categories).
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isgraph</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all codepoints above
		  <symbol>SPACE</symbol> which are not
		  <function>unicode_isspace</function>().
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_islower</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <function>unicode_isalpha</function>() for which the
		  character is
		  equal to
		  <link linkend="unicode_uc">
		    <citerefentry><refentrytitle>unicode_lc</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>
		  of itself.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_ispunct</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <symbol>UNICODE_CATEGORY_1_PUNCTUATION</symbol>.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isspace</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for unicode_isblank() or
		  for unicode characters
		  with linebreaking properties of
		  <symbol>BK</symbol>,
		  <symbol>CR</symbol>,
		  <symbol>LF</symbol>,
		  <symbol>NL</symbol>,
		  and
		  <symbol>SP</symbol>.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
              <term><function>unicode_isupper</function>()</term>
	      <listitem>
		<para>
		  Returns non-0 for all
		  <function>unicode_isalpha</function>() for which the
		  character is
		  equal to
		  <link linkend="unicode_uc">
		    <citerefentry><refentrytitle>unicode_uc</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>
		  of itself.
		</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	</refsect1>
	<refsect1 id="unicode_category_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_convert">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_convert</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_u_ucs4_native</refname>
	  <refname>unicode_u_ucs2_native</refname>
	  <refname>unicode_convert_init</refname>
	  <refname>unicode_convert</refname>
	  <refname>unicode_convert_deinit</refname>

          <refname>unicode_convert_tocbuf_init</refname>
	  <refname>unicode_convert_tou_init</refname>
	  <refname>unicode_convert_fromu_init</refname>
	  <refname>unicode_convert_uc</refname>

	  <refname>unicode_convert_tocbuf_toutf8_init</refname>
	  <refname>unicode_convert_tocbuf_fromutf8_init</refname>

	  <refname>unicode_convert_toutf8</refname>
	  <refname>unicode_convert_fromutf8</refname>
	  <refname>unicode_convert_tobuf</refname>
	  <refname>unicode_convert_tou_tobuf</refname>
	  <refname>unicode_convert_fromu_tobuf</refname>

	  <refpurpose>unicode character set conversion</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;

	    extern const char unicode_u_ucs4_native[];

	    extern const char unicode_u_ucs2_native[];</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_init</function></funcdef>
              <paramdef>const char *<parameter>src_chset</parameter></paramdef>
              <paramdef>const char *<parameter>dst_chset</parameter></paramdef>
              <paramdef>void *<parameter>cb_arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_convert</function></funcdef>
              <paramdef>unicode_convert_handle_t <parameter>handle</parameter></paramdef>
              <paramdef>const char *<parameter>text</parameter></paramdef>
              <paramdef>size_t <parameter>cnt</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_convert_deinit</function></funcdef>
              <paramdef>unicode_convert_handle_t <parameter>handle</parameter></paramdef>
              <paramdef>int *<parameter>errptr</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_tocbuf_init</function></funcdef>
              <paramdef>const char *<parameter>src_chset</parameter></paramdef>
              <paramdef>const char *<parameter>dst_chset</parameter></paramdef>
	      <paramdef>char **<parameter>cbufptr_ret</parameter></paramdef>
	      <paramdef>size_t *<parameter>cbufsize_ret</parameter></paramdef>
              <paramdef>int <parameter>nullterminate</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_tocbuf_toutf8_init</function></funcdef>
              <paramdef>const char *<parameter>src_chset</parameter></paramdef>
	      <paramdef>char **<parameter>cbufptr_ret</parameter></paramdef>
	      <paramdef>size_t *<parameter>cbufsize_ret</parameter></paramdef>
              <paramdef>int <parameter>nullterminate</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_tocbuf_fromutf8_init</function></funcdef>
              <paramdef>const char *<parameter>dst_chset</parameter></paramdef>
	      <paramdef>char **<parameter>cbufptr_ret</parameter></paramdef>
	      <paramdef>size_t *<parameter>cbufsize_ret</parameter></paramdef>
              <paramdef>int <parameter>nullterminate</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_tou_init</function></funcdef>
              <paramdef>const char *<parameter>src_chset</parameter></paramdef>
	      <paramdef>char32_t **<parameter>ucptr_ret</parameter></paramdef>
	      <paramdef>size_t *<parameter>ucsize_ret</parameter></paramdef>
              <paramdef>int <parameter>nullterminate</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_convert_handle_t <function>unicode_convert_fromu_init</function></funcdef>
              <paramdef>const char *<parameter>dst_chset</parameter></paramdef>
	      <paramdef>char **<parameter>cbufptr_ret</parameter></paramdef>
	      <paramdef>size_t *<parameter>cbufsize_ret</parameter></paramdef>
              <paramdef>int <parameter>nullterminate</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_convert_uc</function></funcdef>
              <paramdef>unicode_convert_handle_t <parameter>handle</parameter></paramdef>
              <paramdef>const char32_t *<parameter>text</parameter></paramdef>
	      <paramdef>size_t <parameter>cnt</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>char *<function>unicode_convert_toutf8</function></funcdef>
              <paramdef>const char *<parameter>text</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
	      <paramdef>int *<parameter>error</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>char *<function>unicode_convert_fromutf8</function></funcdef>
              <paramdef>const char *<parameter>text</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
	      <paramdef>int *<parameter>error</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>char *<function>unicode_convert_tobuf</function></funcdef>
              <paramdef>const char *<parameter>text</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
              <paramdef>const char *<parameter>dstcharset</parameter></paramdef>
	      <paramdef>int *<parameter>error</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_convert_toubuf</function></funcdef>
              <paramdef>const char *<parameter>text</parameter></paramdef>
	      <paramdef>size_t <parameter>text_l</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
              <paramdef>char32_t **<parameter>uc</parameter></paramdef>
	      <paramdef>size_t *<parameter>ucsize</parameter></paramdef>
	      <paramdef>int *<parameter>error</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_convert_fromu_tobuf</function></funcdef>
              <paramdef>const char32_t *<parameter>utext</parameter></paramdef>
	      <paramdef>size_t <parameter>utext_l</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
              <paramdef>char **<parameter>c</parameter></paramdef>
	      <paramdef>size_t *<parameter>csize</parameter></paramdef>
	      <paramdef>int *<parameter>error</parameter></paramdef>
	    </funcprototype>

	  </funcsynopsis>

	</refsynopsisdiv>

	<refsect1 id="unicode_convert_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    <varname>unicode_u_ucs4_native</varname>[] contains the
	    string <quote>UCS-4BE</quote> or <quote>UCS-4LE</quote>,
	    matching the native <classname>char32_t</classname> endianness.
	  </para>

	  <para>
	    <varname>unicode_u_ucs2_native</varname>[] contains the
	    string <quote>UCS-2BE</quote> or <quote>UCS-2LE</quote>,
	    matching the native <classname>char32_t</classname> endianness.
	  </para>

	  <para>
	    <function>unicode_convert_init</function>(),
	    <function>unicode_convert</function>(), and
	    <function>unicode_convert_deinit</function>() are an adaption of th
	    <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></ulink> API that uses the same
	      calling convention as the other algorithms in this unicode library,
	      with some value-added features.
	      These functions use
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry> to effect the actual
	      character set conversion.
	  </para>

	  <para>
	    <function>unicode_convert_init</function>() returns a non-NULL handle
	    for the requested conversion, or NULL if the requested conversion is
	    not available.

	    <function>unicode_convert_init</function>() takes a pointer to
	    the output function that receives receives converted character text.
	    The output function receives a pointer to the converted character
	    text, and the number of characters in the converted text.
	    The output function gets repeatedly called, until it receives
	    the entire converted text.
	  </para>

	  <para>
	    The character text to convert gets passed, repeatedly, to
	    <function>unicode_convert</function>().
	    Each call to <function>unicode_convert</function>() results in
	    the output function getting invoked, zero or more times, with each
	    successive part of the
	    converted text. Finally,
	    <function>unicode_convert_deinit</function>() stops the conversion
	    and deallocates the conversion handle.
	  </para>

	  <para>
	    It's possible that a call to
	    <function>unicode_convert_deinit</function>() results in some
	    additional calls to the output function, passing the remaining,
	    final parts, of the converted text, before
	    <function>unicode_convert_deinit</function>() deallocates the
	    handle, and returns.
	  </para>

	  <para>
	    The output function should return 0 normally. A non-0 return
	    indicates n error condition.
	    <function>unicode_convert_deinit</function>() returns
	    non-zero if any previous invocation of the output function returned
	    non-zero (this includes any invocations of the output function
	    resulting from this call, or prior
	    <function>unicode_convert</function>() calls), or 0 if all
	    invocations of the output function returned 0.
	  </para>

	  <para>
	    If the <parameter>errptr</parameter> is not <literal>NULL</literal>,
	    *<parameter>errptr</parameter> gets set to non-zero if there were
	    any conversion errors -- if there was any text that could not be
	    converted to the destination character text.
	  </para>

	  <para>
	    <function>unicode_convert</function>() also returns non-zero if
	    it calls the output function and it returns non-zero, however
	    the conversion handle remains allocated, so
	    <function>unicode_convert_deinit</function>() must still be called,
	    to clean that up.
	  </para>


	  <refsect2 id="unicode_convert_collect">
	    <title>Collecting converted text into a buffer</title>

	    <para>
	      Call
	      <function>unicode_convert_tocbuf_init</function>()
	      instead of
	      <function>unicode_convert_init</function>(), then call
	      <function>unicode_convert</function>()
	      and
	      <function>unicode_convert_deinit</function>() normally.

	      The parameters to
	      <function>unicode_convert_init</function>() specify the source
	      and the destination character sets.

	      <function>unicode_convert_tocbuf_toutf8_init</function>() is
	      just an alias that specifies <literal>UTF-8</literal> as the
	      destination character set.

	      <function>unicode_convert_tocbuf_fromutf8_init</function>() is
	      just an alias that specifies <literal>UTF-8</literal> as the
	      source character st.

	    </para>

	    <para>
	      These functions
	      supply an output function that collects the converted text into
	      a malloc()ed buffer.

	      If <function>unicode_convert_deinit</function>() returns 0,
	      *<parameter>cbufptr_ret</parameter> gets initialized to a
	      malloc()ed buffer, and the number of converted characters,
	      the size of the
	      malloc()ed buffer, get placed into
	      *<parameter>cbufsize_ret</parameter>.
	    </para>

	    <note>
	      <para>
		If the converted string is an empty string,
		*<parameter>cbufsize_ret</parameter>
		gets set to 0,
		but
		*<parameter>cbufptr_ret</parameter>
		still gets initialized (to a dummy malloced buffer).
	      </para>
	    </note>

	    <para>
	      A non-zero <parameter>nullterminate</parameter> places a trailing \0
	      character after the converted string (this is included in
	      *<parameter>cbufsize_ret</parameter>).
	    </para>
	  </refsect2>

	  <refsect2 id="unicode_convert_chset_unicode">
	    <title>Converting between character sets and unicode</title>

	    <para>
	      <function>unicode_convert_tou_init</function>()
	      converts character text into a <classname>char32_t</classname>
	      buffer.
	      It works just like
	      <function>unicode_convert_tocbuf_init</function>(), except
	      that only the source character set gets specified and the output
	      buffer is a <classname>char32_t</classname> buffer.
	      <parameter>nullterminate</parameter> terminates the converted
	      unicode characters with a <literal>U+0000</literal>.
	    </para>

	    <para>
	      <function>unicode_convert_fromu_init</function>()
	      converts <classname>char32_t</classname>s to the output
	      character set, and also works like
	      <function>unicode_convert_tocbuf_init</function>().
	      Additionally, in this case,
	      <function>unicode_convert_uc</function>() works just like
	      <function>unicode_convert</function>() except that the
	      input sequence is a
	      <classname>char32_t</classname> sequence, and the
	      count parameter is th enumber of unicode characters.
	    </para>
	  </refsect2>

	  <refsect2 id="unicode_convert_oneshot">
	    <title>One-shot conversions</title>

	    <para>
	      <function>unicode_convert_toutf8</function>()
	      converts the specified text in the specified text into a UTF-8
	      string, returning a malloced buffer.
	      If <parameter>error</parameter> is
	      not <literal>NULL</literal>, even if
	      <function>unicode_convert_toutf8</function>()
	      returns a non <literal>NULL</literal> value
	      *<parameter>error</parameter> gets set to a non-zero value if
	      a character conversion error has occured, and some characters
	      could not be converted.
	    </para>

	    <para>
	      <function>unicode_convert_fromutf8</function>() does a similar
	      conversion from UTF-8 <parameter>text</parameter> to the specified
	      character set.
	    </para>

	    <para>
	      <function>unicode_convert_tobuf</function>() does a similar
	      conversion between two different character sets.
	    </para>

	    <para>
	      <function>unicode_convert_tou_tobuf</function>() calls
	      <function>unicode_convert_tou_init</function>(), feeds the
	      character string through <function>unicode_convert</function>(),
	      then calls <function>unicode_convert_deinit</function>().
	      If this function returns 0,
	      *<parameter>uc</parameter> and *<parameter>ucsize</parameter>
	      are set to a malloced buffer+size holding the unicode char array.
	    </para>

	    <para>
	      <function>unicode_convert_fromu_tobuf</function>() calls
	      <function>unicode_convert_fromu_init</function>(), feeds the
	      unicode array through <function>unicode_convert_uc</function>(),
	      then calls unicode_convert_deinit().

	      If this function returns 0,
	      *<parameter>c</parameter> and *<parameter>csize</parameter>
	      are set to a malloced buffer+size holding the char array.
	    </para>
	  </refsect2>
	</refsect1>
	<refsect1 id="unicode_convert_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <link linkend="unicode_default_chset">
		<citerefentry><refentrytitle>unicode_default_chset</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_default_chset">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_default_chset</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_default_chset</refname>
	  <refname>unicode_locale_chset</refname>
	  <refpurpose>return the system character set name</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>const char *<function>unicode_default_chset</function></funcdef>
	      <void />
	    </funcprototype>

	    <funcprototype>
              <funcdef>const char *<function>unicode_locale_chset</function></funcdef>
	      <void />
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_default_chset_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    <function>unicode_default_chset</function>() returns the name of the
	    system environment character set (usually
	    <quote>nl_langinfo(CODESET)</quote>, or from some suitable environment
	    variable).
	    <function>unicode_locale_chset</function>() returns the name of the
	    current application locale's character set.
	  </para>
	</refsect1>
	<refsect1 id="unicode_default_chset_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_emoji_lookup">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_emoji_lookup</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_emoji_lookup</refname>
          <refname>unicode_emoji</refname>
          <refname>unicode_emoji_presentation</refname>
	  <refname>unicode_emoji_modifier</refname>
	  <refname>unicode_emoji_modifier_base</refname>
	  <refname>unicode_emoji_component</refname>
	  <refname>unicode_emoji_extended_pictographic</refname>

	  <refpurpose>look up unicode character's Unicode Emoji Classification</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>unicode_emoji_t <function>unicode_emoji_lookup</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji_presentation</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji_modifier</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji_modifier_base</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji_component</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>int <function>unicode_emoji_extended_pictographic</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_emoji_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    <function>unicode_emoji_lookup</function>() returns the
	    unicode emoji properties of the specified character, as a bitmask
	    of <literal>UNICODE_EMOJI</literal> flags, as defined in the
	    header file.
	    <function>unicode_emoji</function>(),
	    <function>unicode_emoji_presentation</function>(),
	    <function>unicode_emoji_modifier</function>(),
	    <function>unicode_emoji_modifier_base</function>(),
	    <function>unicode_emoji_component</function>(), and
	    <function>unicode_emoji_extended_pictographic</function>()
	    check whether the given character carries a specific emoji
	    property. They return 0 if not, and non-0 if the specified
	    character has the corresponding property.
	  </para>
        </refsect1>
	<refsect1 id="unicode_emoji_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <ulink url="https://www.unicode.org/reports/tr51/tr51-&tr51ver;.html">TR-51</ulink>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_html40ent_lookup">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode_html40ent_lookup</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_html40ent_lookup</refname>
	  <refpurpose>look up unicode character for an HTML 4.0 entity</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>char32_t <function>unicode_html40ent_lookup</function></funcdef>
              <paramdef>const char *<parameter>entity</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_html40_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    <function>unicode_html40ent_lookup</function>() returns the
	    unicode character represented by an HTML 4.0 entity. The
	    <parameter>entity</parameter> is a string, such as
	    <quote>quot</quote>, in which case
	    <function>unicode_html40ent_lookup</function>() returns 34.
	  </para>

	  <para>
	    Additionally,
	    <function>unicode_html40ent_lookup</function>() parses
	    a numerical entity given as
	    <quote>#<replaceable>decimal</replaceable></quote> or
	    <quote>#x<replaceable>hex</replaceable></quote>.
	  </para>

	  <para>
	    <function>unicode_html40ent_lookup</function>() returns 0 if the
	    <parameter>entity</parameter> is not a known entity that represents
	    a single unicode character.
	  </para>
	</refsect1>
	<refsect1 id="unicode_html40_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_grapheme_break">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>unicode_grapheme_break</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_grapheme_break</refname>
	  <refname>unicode_grapheme_break_init</refname>
	  <refname>unicode_grapheme_break_next</refname>
	  <refname>unicode_grapheme_break_deinit</refname>
	  <refpurpose>unicode grapheme cluster boundary rules</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>

	    <funcprototype>
              <funcdef>unicode_grapheme_break_info_t <function>unicode_grapheme_break_init</function></funcdef>
	      <void />
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_grapheme_next</function></funcdef>
              <paramdef>unicode_grapheme_break_info_t <parameter>handle</parameter></paramdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode_grapheme_deinit</function></funcdef>
              <paramdef>unicode_grapheme_break_info_t <parameter>handle</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_grapheme_break</function></funcdef>
              <paramdef>char32_t <parameter>a</parameter></paramdef>
              <paramdef>char32_t <parameter>b</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_grapheme_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions implement the unicode grapheme cluster breaking
	    algorithm. Invoke
	    <function>unicode_grapheme_break_init</function>() to initialize
	    the grapheme cluster breaking algorithm.
	    <function>unicode_grapheme_break_init</function>() returns an
	    opaque handle. Each subsequent call to
	    <function>unicode_grapheme_break_next</function>() passes this
	    handle, and the next character.
	    <function>unicode_grapheme_break_next</function>() returns a non-0
	    value if there's a grapheme break before the character, in a
	    sequence of Unicode characters.
	    <function>unicode_grapheme_break_deinit</function>() releases
	    all reosurces used by the grapheme breaking handle, and the
	    <classname>unicode_grapheme_break_info_t</classname> handle
	    is no longer valid after this call.
	  </para>
	  <para>
	    The first call to <function>unicode_grapheme_break_next</function>()
	    always returns non-0, as per the GB1 rule.
	  </para>
	  <para>
	    <function>unicode_grapheme_break</function>() is a simplified
	    interface that returns non-zero
	    if there is a grapheme break between two unicode characters
	    <parameter>a</parameter> and
	    <parameter>b</parameter>.
	    This is is equivalent to calling
	    <function>unicode_grapheme_break_init</function>(),
	    followed by two calls to
	    <function> unicode_grapheme_break_next</function>(), and finally
	    <function>unicode_grapheme_break_deinit</function>(), then
	    returning
	    the result of the second
	    call to <function>unicode_grapheme_break_next</function>().
	  </para>
	</refsect1>

	<refsect1 id="unicode_grapheme_seealso">
	  <title>SEE ALSO</title>

	  <para>
	    <ulink url="https://www.unicode.org/reports/tr29/tr29-&tr29ver;.html">TR-29</ulink>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_uc">
	      <citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <link linkend="unicode_line_break">
		<citerefentry><refentrytitle>unicode_line_break</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>,
		<link linkend="unicode_word_break">
		  <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_line_break">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>unicode_line_break</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_line_break</refname>
	  <refname>unicode_lb_init</refname>
	  <refname>unicode_lb_set_opts</refname>
	  <refname>unicode_lb_next</refname>
	  <refname>unicode_lb_next_cnt</refname>
	  <refname>unicode_lb_end</refname>

	  <refname>unicode_lbc_init</refname>
	  <refname>unicode_lbc_set_opts</refname>
	  <refname>unicode_lbc_next</refname>
	  <refname>unicode_lbc_next_cnt</refname>
	  <refname>unicode_lbc_end</refname>
	  <refpurpose>calculate mandatory or allowed line breaks</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>unicode_lb_info_t <function>unicode_lb_init</function></funcdef>
              <paramdef>int (*<parameter>cb_func</parameter>)(int, void *)</paramdef>
              <paramdef>void *<parameter>cb_arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode_lb_set_opts</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
              <paramdef>int <parameter>opts</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lb_next</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lb_next_cnt</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
              <paramdef>const char32_t *<parameter>cptr</parameter></paramdef>
              <paramdef>size_t <parameter>cnt</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lb_end</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>unicode_lbc_info_t <function>unicode_lbc_init</function></funcdef>
              <paramdef>int (*<parameter>cb_func</parameter>)(int, char32_t, void *)</paramdef>
              <paramdef>void *<parameter>cb_arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode_lbc_set_opts</function></funcdef>
              <paramdef>unicode_lbc_info_t <parameter>lb</parameter></paramdef>
              <paramdef>int <parameter>opts</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lbc_next</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lbc_next_cnt</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
              <paramdef>const char32_t *<parameter>cptr</parameter></paramdef>
              <paramdef>size_t <parameter>cnt</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_lbc_end</function></funcdef>
              <paramdef>unicode_lb_info_t <parameter>lb</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_lb_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    These functions implement the unicode line breaking algorithm.
	    Invoke <function>unicode_lb_init</function>() to initialize the
	    line breaking algorithm. The
	    first parameter is a callback function.
	    The second parameter is an opaque pointer.
	    The callback function gets invoked with two parameters. The first
	    parameter is one of three values:
	    <literal>UNICODE_LB_MANDATORY</literal>,
	    <literal>UNICODE_LB_NONE</literal>, or
	    <literal>UNICODE_LB_ALLOWED</literal>, as described below.
	    The second parameter is
	    the opaque pointer that was passed to
	    <function>unicode_lb_init</function>(); the opaque pointer
	    is not subject to any further interpretation by these functions.
	  </para>

	  <para>
	    <function>unicode_lb_init</function>() returns an opaque handle.
	    Repeated invocations of <function>unicode_lb_next</function>(),
	    passing the handle and one unicode character at a time,
	    defines a sequence
	    of unicode characters over which the line breaking algorithm
	    calculation takes place.
	    <function>unicode_lb_next_cnt</function>() is a shortcut
	    for invoking <function>unicode_lb_next</function>()
	    repeatedly over an array
	    <literal>cptr</literal> containing
	    <literal>cnt</literal> unicode characters.
	  </para>

	  <para>
	    <function>unicode_lb_end</function>() denotes the end of the
	    unicode character sequence. After the call to
	    <function>unicode_lb_end</function>() the line breaking
	    <classname>unicode_lb_info_t</classname> handle is no longer valid.
	  </para>

	  <para>
	    Between the call to
	    <function>unicode_lb_init</function>() and
	    <function>unicode_lb_end</function>(), the callback function
	    gets invoked exactly once for each unicode character given to
	    <function>unicode_lb_next</function>() or
	    <function>unicode_lb_next_cnt</function>().
	    Usually each call to
	    <function>unicode_lb_next</function>() results in the callback
	    function getting invoked immediately, but it does not have to be.
	    It's possible that a call to <function>unicode_lb_next</function>()
	    returns without invoking the callback function, and some subsequent
	    call to <function>unicode_lb_next</function>() (or
	    <function>unicode_lb_end</function>()) invokes the callback function
	    more than once, to catch up.
	    The contract is that before <function>unicode_lb_end</function>()
	    returns, the callback function gets invoked the exact number of times
	    as the number of characters in the unicode sequence defined by
	    the intervening calls to <function>unicode_lb_next</function>() and
	    <function>unicode_lb_next_cnt</function>(), unless an error
	    occurs.
	  </para>

	  <para>
	    Each call to the callback function reports the calculated
	    line breaking status of the corresponding character in the unicode
	    character sequence:
	  </para>

	  <variablelist>
	    <varlistentry>
	      <term><literal>UNICODE_LB_MANDATORY</literal></term>
	      <listitem>
		<para>
		  A line break is MANDATORY
		  <emphasis>before</emphasis> the corresponding character.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><literal>UNICODE_LB_NONE</literal></term>
	      <listitem>
		<para>
		  A line break is PROHIBITED
		  <emphasis>before</emphasis> the corresponding character.
		</para>
	      </listitem>
	    </varlistentry>

	    <varlistentry>
	      <term><literal>UNICODE_LB_ALLOWED</literal></term>
	      <listitem>
		<para>
		  A line break is OPTIONAL
		  <emphasis>before</emphasis> the corresponding character.
		</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <para>
	    The callback function should return 0. A non-zero value
	    indicates to the line breaking algorithm that an error has
	    occured.
	    <function>unicode_lb_next</function>() and
	    <function>unicode_lb_next_cnt</function>() return zero either if
	    they never invoked the callback function, or if each call to the
	    callback function returned zero.
	    A non zero return from the callback function results in
	    <function>unicode_lb_next</function>() and
	    <function>unicode_lb_next_cnt</function>() immediately returning
	    the same value.
	  </para>

	  <para>
	    <function>unicode_lb_end</function>() must be invoked to destroy
	    the line breaking handle even if
	    <function>unicode_lb_next</function>() and
	    <function>unicode_lb_next_cnt</function>() returned an error
	    indication. It's also possible that, under normal circumstances,
	    <function>unicode_lb_end</function>() invokes the callback function
	    one or more times. The return value from
	    <function>unicode_lb_end</function>() has the same meaning as
	    from <function>unicode_lb_next</function>() and
	    <function>unicode_lb_next_cnt</function>(); however in all cases
	    after <function>unicode_lb_end</function>() returns the
	    line breaking handle is no longer valid.
	  </para>

	  <refsect2 id="unicode_lb_altcallback">
	    <title>Alternative callback function</title>

	    <para>
	      <function>unicode_lbc_init</function>(),
	      <function>unicode_lbc_next</function>(),
	      <function>unicode_lbc_next_cnt</function>(),
	      <function>unicode_lbc_end</function>() are alternative functions
	      that implement the same algorithm. The only difference is that
	      the callback function receives an extra parameter, the unicode
	      character value to which the line breaking status applies to,
	      passed through from the input unicode character sequence.
	    </para>
	  </refsect2>

	  <refsect2 id="unicode_lb_altcallback_opt">
	    <title>Options</title>

	    <para>
	      <function>unicode_lb_set_opts</function>() and
	      <function>unicode_lbc_set_opts</function>() enable non-default
	      options for the line breaking algorithm. These functions must be
	      called immediately after
	      <function>unicode_lb_init</function>() or
	      <function>unicode_lbc_init</function>(), and before any other
	      function.
	      <parameter>opts</parameter> is a bitmask that can contain
	      the following values:
	    </para>

	    <variablelist>
	      <varlistentry>
		<term><literal>UNICODE_LB_OPT_PRBREAK</literal></term>
		<listitem>
		  <para>
		    Enables a modified <literal>LB24</literal> rule.
		    This prevents
		    plus signs, as in <quote>C++</quote> from breaking. This
		    flag adds the following rules to the LB24 rule:
		  </para>

		  <blockquote>
		    <informalexample>
		      <programlisting>
			PR x PR

			AL x PR

		        ID x PR</programlisting>
		    </informalexample>
		  </blockquote>
		</listitem>
	      </varlistentry>
	      <varlistentry>
		<term><literal>UNICODE_LB_OPT_SYBREAK</literal></term>
		<listitem>
		  <para>
		    Tailored breaking rules for the <quote>/</quote> character.
		    This prevents breaking after the <quote>/</quote> character
		    (think URLs); including an exception to the
		    <quote>x SY</quote> rule in <literal>LB13</literal>.
		    This flag adds the following rules to the LB24 rule:
		  </para>
		  <blockquote>
		    <informalexample>
		      <programlisting>
			SY x EX

			SY x AL

			SY x ID

		        SP ÷ SY, which takes precedence over "x SY".</programlisting>
		    </informalexample>
		  </blockquote>
		</listitem>
	      </varlistentry>
	      <varlistentry>
		<term><literal>UNICODE_LB_OPT_DASHWJ</literal></term>
		<listitem>
		  <para>
		    This flag reclassifies <literal>U+2013</literal> and
		    <literal>U+2014</literal> as class <literal>WJ</literal>,
		    prohibiting breaks before and after the m-dash and the
		    n-dash unicode characters.
		  </para>
		</listitem>
	      </varlistentry>
	    </variablelist>
	  </refsect2>
	</refsect1>

	<refsect1 id="unicode_lb_seealso">
	  <title>SEE ALSO</title>

	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode__linebreak">
	      <citerefentry><refentrytitle>unicode::linebreak</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <ulink url="https://www.unicode.org/reports/tr14/tr14-&tr14ver;.html">TR-14</ulink>
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_script">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>unicode_script</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_script</refname>
	  <refpurpose>unicode script property</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>unicode_script_t <function>unicode_script</function></funcdef>
              <paramdef>char32_t <parameter>ch</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_script_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    <function>unicode_script</function>() looks up the
	    <quote>script</quote> property of the specified unicode character,
	    and returns it. The <classname>unicode_script_t</classname>
	    enumeration encodes possible unicode script values.
	    <literal>unicode_script_unknown</literal> gets returned for a
	    unicode character  with an unknown script property.
	  </para>
	</refsect1>

	<refsect1 id="unicode_script_seealso">
	  <title>SEE ALSO</title>

	  <para>
	    <ulink url="https://www.unicode.org/reports/tr24/tr24-&tr24ver;.html">TR-24</ulink>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_word_break">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>unicode_word_break</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_wb_init</refname>
	  <refname>unicode_wb_next</refname>
	  <refname>unicode_wb_next_cnt</refname>
	  <refname>unicode_wb_end</refname>

	  <refname>unicode_wbscan_init</refname>
	  <refname>unicode_wbscan_next</refname>
	  <refname>unicode_wbscan_end</refname>
	  <refname>unicode_word_break</refname>

	  <refpurpose>calculate word breaks</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>unicode_wb_info_t <function>unicode_wb_init</function></funcdef>
              <paramdef>int (*<parameter>cb_func</parameter>)(int, void *)</paramdef>
              <paramdef>void *<parameter>cb_arg</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_wb_next</function></funcdef>
              <paramdef>unicode_wb_info_t <parameter>wb</parameter></paramdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_wb_next_cnt</function></funcdef>
              <paramdef>unicode_wb_info_t <parameter>wb</parameter></paramdef>
              <paramdef>const char32_t *<parameter>cptr</parameter></paramdef>
              <paramdef>size_t <parameter>cnt</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_wb_end</function></funcdef>
              <paramdef>unicode_wb_info_t <parameter>wb</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>unicode_wbscan_info_t <function>unicode_wbscan_init</function></funcdef>
	      <void />
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode_wbscan_next</function></funcdef>
              <paramdef>unicode_wbscan_info_t <parameter>wbs</parameter></paramdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>size_t <function>unicode_wbscan_end</function></funcdef>
              <paramdef>unicode_wbscan_info_t <parameter>wbs</parameter></paramdef>
	    </funcprototype>

	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_wb_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    These functions implement the unicode word breaking algorithm.
	    Invoke <function>unicode_wb_init</function>() to initialize the
	    word breaking algorithm. The
	    first parameter is a callback function.
	    The second parameter is an opaque pointer.
	    The callback function gets invoked with two parameters.
	    The second parameter is
	    the opaque pointer that was given to
	    <function>unicode_wb_init</function>(); and the opaque pointer
	    is not subject to any further interpretation by these functions.
	  </para>

	  <para>
	    <function>unicode_wb_init</function>() returns an opaque handle.
	    Repeated invocations of <function>unicode_wb_next</function>(),
	    passing the handle, and one unicode character defines a sequence
	    of unicode characters over which the word breaking algorithm
	    calculation takes place.
	    <function>unicode_wb_next_cnt</function>() is a shortcut
	    for invoking <function>unicode_wb_next</function>()
	    repeatedly over an array
	    <literal>cptr</literal> containing
	    <literal>cnt</literal> unicode characters.
	  </para>

	  <para>
	    <function>unicode_wb_end</function>() denotes the end of the
	    unicode character sequence. After the call to
	    <function>unicode_wb_end</function>() the word breaking
	    <classname>unicode_wb_info_t</classname> handle is no longer valid.
	  </para>

	  <para>
	    Between the call to
	    <function>unicode_wb_init</function>() and
	    <function>unicode_wb_end</function>(), the callback function
	    gets invoked exactly once for each unicode character given to
	    <function>unicode_wb_next</function>() or
	    <function>unicode_wb_next_cnt</function>().
	    Usually each call to
	    <function>unicode_wb_next</function>() results in the callback
	    function getting invoked immediately, but it does not have to be.
	    It's possible that a call to <function>unicode_wb_next</function>()
	    returns without invoking the callback function, and some subsequent
	    call to <function>unicode_wb_next</function>() (or
	    <function>unicode_wb_end</function>()) invokes the callback function
	    more than once, to catch things up.
	    The contract is that before <function>unicode_wb_end</function>()
	    returns, the callback function gets invoked the exact number of times
	    as the number of characters in the unicode sequence defined by
	    the intervening calls to <function>unicode_wb_next</function>() and
	    <function>unicode_wb_next_cnt</function>(), unless an error
	    occurs.
	  </para>

	  <para>
	    Each call to the callback function reports the calculated
	    wordbreaking status of the corresponding character in the unicode
	    character sequence. If the parameter to the callback function
	    is non zero, a word break is permitted <emphasis>before</emphasis>
	    the corresponding character. A zero value indicates that a word
	    break is prohibited <emphasis>before</emphasis> the corresponding
	    character.
	  </para>

	  <para>
	    The callback function should return 0. A non-zero value
	    indicates to the word breaking algorithm that an error has
	    occured.
	    <function>unicode_wb_next</function>() and
	    <function>unicode_wb_next_cnt</function>() return zero either if
	    they never invoked the callback function, or if each call to the
	    callback function returned zero.
	    A non zero return from the callback function results in
	    <function>unicode_wb_next</function>() and
	    <function>unicode_wb_next_cnt</function>() immediately returning
	    the same value.
	  </para>

	  <para>
	    <function>unicode_wb_end</function>() must be invoked to destroy
	    the word breaking handle even if
	    <function>unicode_wb_next</function>() and
	    <function>unicode_wb_next_cnt</function>() returned an error
	    indication. It's also possible that, under normal circumstances,
	    <function>unicode_wb_end</function>() invokes the callback function
	    one or more times. The return value from
	    <function>unicode_wb_end</function>() has the same meaning as
	    from <function>unicode_wb_next</function>() and
	    <function>unicode_wb_next_cnt</function>(); however in all cases
	    after <function>unicode_wb_end</function>() returns the
	    line breaking handle is no longer valid.
	  </para>

	  <refsect2 id="unicode_wb_scan">
	    <title>Word scan</title>

	    <para>
	      <function>unicode_wbscan_init</function>(),
              <function>unicode_wbscan_next</function>() and
              <function>unicode_wbscan_end</function>
	      scan for the next word boundary in a unicode character sequence.

	      <function>unicode_wbscan_init</function>() obtains a handle,
	      then
              <function>unicode_wbscan_next</function>() gets repeatedly invoked
	      to define the unicode character sequence.
	      <function>unicode_wbscan_end</function>() deallocates the handle
	      and returns the number of leading characters in the unicode character
	      sequence up to the first word break.
	    </para>

	    <para>
	      A non-0 return value from
              <function>unicode_wbscan_next</function>() indicates that the
	      word boundary is already known, and any further calls to
              <function>unicode_wbscan_next</function>() will be ignored.
	      <function>unicode_wbscan_end</function>() must still be called,
	      to obtain the unicode character count.
	    </para>
	  </refsect2>
	</refsect1>

	<refsect1 id="unicode_wb_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <ulink url="https://www.unicode.org/reports/tr29/tr29-&tr29ver;.html">TR-29</ulink>,
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode::wordbreak</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <link linkend="unicode_uc">
		<citerefentry><refentrytitle>unicode_convert_tocase</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>,
		<link linkend="unicode_line_break">
		  <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>,
		  <link linkend="unicode_grapheme_break">
		    <citerefentry><refentrytitle>unicode_grapheme_break</refentrytitle>
		    <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode_uc">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>
	<refmeta>
	  <refentrytitle>unicode_uc</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode_uc</refname>
	  <refname>unicode_lc</refname>
	  <refname>unicode_tc</refname>
	  <refname>unicode_convert_tocase</refname>
	  <refpurpose>unicode uppercase, lowercase, and titlecase character lookup</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>char32_t <function>unicode_uc</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	  <funcsynopsis>
	    <funcprototype>
              <funcdef>char32_t <function>unicode_lc</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	  <funcsynopsis>
	    <funcprototype>
              <funcdef>char32_t <function>unicode_tc</function></funcdef>
              <paramdef>char32_t <parameter>c</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	  <funcsynopsis>
	    <funcprototype>
              <funcdef>char *<function>unicode_convert_tocase</function></funcdef>
              <paramdef>const char *<parameter>str</parameter></paramdef>
              <paramdef>const char *<parameter>charset</parameter></paramdef>
              <paramdef>char32_t (*<parameter>first_char_func</parameter>)(uncode_char)</paramdef>
              <paramdef>char32_t (*<parameter>char_func</parameter>)(uncode_char)</paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>
	<refsect1 id="unicode_uc_descr">
	  <title>DESCRIPTION</title>
	  <para>
	    <function>unicode_uc</function>(),
	    <function>unicode_lc</function>(),
	    <function>unicode_tc</function>()
	    return the uppercase, lowercase, or the titlecase
	    equivalent of the unicode character <parameter>c</parameter>.
	    If this character does not have an uppercase, lowercase, or a titlecase
	    equivalent, these functions return <parameter>c</parameter>, the
	    same character.
	  </para>

	  <para>
	    <function>unicode_convert_tocase</function>()
	    takes the string <parameter>str</parameter> in the
	    character set <parameter>charset</parameter>.
	    <parameter>first_char_func</parameter> and
	    <parameter>char_func</parameter>, each, should be
	    <function>unicode_uc</function>,
	    <function>unicode_lc</function>, or
	    <function>unicode_tc</function>.
	    <function>unicode_convert_tocase</function>() returns a malloc()ed
	    buffer. The first unicode character in
	    <parameter>str</parameter> gets processed by
	    <parameter>first_char_func</parameter>, and all other characters by
	    <parameter>char_func</parameter>.
	  </para>
	</refsect1>

	<refsect1 id="unicode_uc_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry>
		<refentrytitle>unicode_convert</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>,
	      <link linkend="unicode_default_chset">
		<citerefentry><refentrytitle>unicode_default_chset</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>,
		<link linkend="unicode_html40ent_lookup">
		  <citerefentry><refentrytitle>unicode_html40ent_lookup</refentrytitle>
		  <manvolnum>3</manvolnum></citerefentry></link>,
		  <link linkend="unicode_category_lookup">
		    <citerefentry><refentrytitle>unicode_category_lookup</refentrytitle>
		    <manvolnum>3</manvolnum></citerefentry></link>,
		    <link linkend="unicode_grapheme_break">
		      <citerefentry><refentrytitle>unicode_grapheme_break</refentrytitle>
		      <manvolnum>3</manvolnum></citerefentry></link>,
		      <link linkend="unicode_word_break">
			<citerefentry><refentrytitle>unicode_word_break</refentrytitle>
			<manvolnum>3</manvolnum></citerefentry></link>,
			<link linkend="unicode_line_break">
			  <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
			  <manvolnum>3</manvolnum></citerefentry></link>.

	  </para>
	</refsect1>
      </refentry>
    </section>

    <section id="manpagescpp">
      <title>C++ manual pages</title>

      <refentry id="unicode__bidi">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::bidi</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::bidi</refname>
	  <refname>unicode::bidi_calc</refname>
	  <refname>unicode::bidi_calc_types</refname>
	  <refname>unicode::bidi_reorder</refname>
	  <refname>unicode::bidi_cleanup</refname>
	  <refname>unicode::bidi_logical_order</refname>
	  <refname>unicode::bidi_combinings</refname>
	  <refname>unicode::bidi_needs_embed</refname>
	  <refname>unicode::bidi_embed</refname>
	  <refname>unicode::bidi_embed_paragraph_level</refname>
	  <refname>unicode::bidi_get_direction</refname>
	  <refname>unicode::bidi_override</refname>
	  <refpurpose>unicode bi-directional algorithm</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <synopsis>#include &lt;courier-unicode.h&gt;</synopsis>
	  <classsynopsis class="class" language="C++">
	    <ooclass>
	      <classname>struct unicode::bidi_calc_types</classname>
	    </ooclass>
	    <constructorsynopsis>
	      <methodname>bidi_calc_types</methodname>
	      <methodparam><modifier>const std::u32string &amp;</modifier><parameter>string</parameter>
	      </methodparam>
	    </constructorsynopsis>
	    <fieldsynopsis>
	      <modifier>std::vector&lt;unicode_bidi_type_t&gt;</modifier>
	      <varname>types</varname>
	    </fieldsynopsis>

	    <methodsynopsis>
	      <void />
	      <methodname>setbnl</methodname>
	      <methodparam>
		<modifier>std::u32string &amp;</modifier>
		<parameter>string</parameter>
	      </methodparam>
	    </methodsynopsis>
	  </classsynopsis>

	  <funcsynopsis>
	    <funcprototype>
              <funcdef>std::tuple&lt;std::vector&lt;unicode_bidi_level_t&gt;, struct unicode_bidi_direction&gt; <function>unicode::bidi_calc</function></funcdef>
	      <paramdef>const unicode::bidi_calc_types &amp;<parameter>ustring</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::tuple&lt;std::vector&lt;unicode_bidi_level_t&gt;, struct unicode_bidi_direction&gt; <function>unicode::bidi_calc</function></funcdef>
	      <paramdef>const unicode::bidi_calc_types &amp;<parameter>ustring</parameter></paramdef>
	      <paramdef>unicode_bidi_level_t <parameter>embedding_level</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode::bidi_reorder</function></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>std::vector&lt;unicode_bidi_level_t&gt; &amp;<parameter>embedding_level</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t, size_t)&gt; &amp;<parameter>reorder_callback</parameter>=[](size_t, size_t){}</paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::bidi_reorder</function></funcdef>
	      <paramdef>std::vector&lt;unicode_bidi_level_t&gt; &amp;<parameter>embedding_level</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t, size_t)&gt; &amp;<parameter>reorder_callback</parameter>=[](size_t, size_t){}</paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::bidi_cleanup</function></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t)&gt; &amp;<parameter>removed_callback</parameter>=[](size_t){}</paramdef>
	      <paramdef>int <parameter>cleanup_options</parameter></paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode::bidi_cleanup</function></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t)&gt; &amp;<parameter>removed_callback</parameter>=[](size_t){}</paramdef>
	      <paramdef>int <parameter>cleanup_options</parameter>=0</paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode::bidi_cleanup</function></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t)&gt; &amp;<parameter>removed_callback</parameter></paramdef>
	      <paramdef>int <parameter>cleanup_options</parameter></paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter></paramdef>
	      <paramdef>size_t <parameter>n</parameter></paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode::bidi_logical_order</function></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t, size_t)&gt; &amp;<parameter>reorder_callback</parameter>=[](size_t, size_t){}</paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::bidi_combinings</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>const std::function &lt;void (unicode_bidi_level_t level, size_t level_start, size_t n_chars, size_t comb_start, size_t n_comb_chars)&gt; &amp;<parameter>callback</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::bidi_combinings</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::function &lt;void (unicode_bidi_level_t level, size_t level_start, size_t n_chars, size_t comb_start, size_t n_comb_chars)&gt; &amp;<parameter>callback</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::bidi_logical_order</function></funcdef>
	      <paramdef>std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
	      <paramdef>const std::function&lt;void (size_t, size_t)&gt; &amp;<parameter>reorder_callback</parameter></paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>bool <function>unicode::bidi_needs_embed</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>const unicode_bidi_level_t (<parameter>paragraph_embedding</parameter>=NULL</paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>int <function>unicode::bidi_embed</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
	      <paramdef>const std::function&lt;void (const char32_t *, size_t, bool)&gt; &amp;<parameter>callback</parameter></paramdef>
            </funcprototype>

	    <funcprototype>
              <funcdef>std::u32string <function>unicode::bidi_embed</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::vector &lt;unicode_bidi_level_t&gt; &amp;<parameter>levels</parameter></paramdef>
	      <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
            </funcprototype>

	    <funcprototype>
	      <funcdef>char32_t <function>unicode_bidi_embed_paragraph_level</function></funcdef>
              <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
              <paramdef>unicode_bidi_level_t <parameter>paragraph_embedding</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>unicode_bidi_direction <function>bidi_get_direction</function></funcdef>
              <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>size_t <parameter>starting_pos</parameter>=0</paramdef>
	      <paramdef>size_t <parameter>n</parameter>=(size_t)-1</paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>std::u32string <function>bidi_override</function></funcdef>
              <paramdef>const std::u32string &amp;<parameter>string</parameter></paramdef>
              <paramdef>unicode_bidi_level_t <parameter>direction</parameter></paramdef>
	      <paramdef>int <parameter>cleanup_options</parameter>=0</paramdef>
	    </funcprototype>
          </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_bidi_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions implement the C++ interface for the
	    <ulink url="https://www.unicode.org/reports/tr9/tr9-&tr9ver;.html"> Unicode Bi-Directional algorithm</ulink>.
	    See the description of the underlying
	    <link linkend="unicode_bidi">
	      <citerefentry><refentrytitle>unicode_bidi</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link> C library
	      API for more information. C++ specific notes:
	  </para>

	  <itemizedlist>
	    <listitem>
	      <para>
                <function>unicode::bidi_calc</function> returns the
		directional embedding value buffer and the calculated paragraph
		embedding level. Its <parameter>ustring</parameter>
		is implicitly converted from a
		<classname>std::u32string</classname>:
              </para>
	      <blockquote>
		<informalexample>
		  <programlisting><![CDATA[
std::u32string text;

auto [levels, direction]=unicode::bidi_calc(text);

]]></programlisting>
		</informalexample>
	      </blockquote>

	      <para>
		Alternatively a <classname>unicode::bidi_calc_types</classname>
		objects gets constructed from the same
		<classname>std::u32string</classname> and then passed
		directly to <function>unicode::bidi_calc</function>:
	      </para>
	      <blockquote>
		<informalexample>
		  <programlisting><![CDATA[
std::u32string text;

unicode::bidi_calc_types types{text};

types.setbnl(text); // Optional

// types.types is a std::vector of enum_bidi_types_t values

auto [levels, direction]=unicode::bidi_calc(types);

]]></programlisting>
		</informalexample>
	      </blockquote>
	      <para>
		This provides the means to access the intermediate
		<classname>enum_bidi_types_t</classname> values that
		get calculated from the Unicode text string.
	      </para>

	      <note>
		<para>
		  In all cases the <classname>std::u32string</classname>
		  cannot be a temporary object, and it must remain in scope
		  until <function>unicode::bidi_calc</function>() returns.
		</para>
	      </note>

	      <para>
		The optional <methodname>setbnl</methodname>() method uses
		<link linkend="unicode_bidi">
		<citerefentry>
		  <refentrytitle>unicode_bidi_setbnl</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>
		to replace paragraph separators with newline characters,
		in the unicode string. It requires the same unicode string
		that was passed to the constructor as a parameter (because
		the constructor takes a constant reference, but this
		method modifies the string.
	      </para>
            </listitem>
	    <listitem>
	      <para>
		Several C functions provide a <quote>dry-run</quote> mode
		by passing a <literal>NULL</literal> pointer. The C++ API
		provides separate overloads, with and without the nullable
		parameter.
              </para>
            </listitem>
	    <listitem>
	      <para>
		Several C functions accept a nullable function pointer, with
		the <literal>NULL</literal> function pointer specifying no
		callback. The C++ functions have a
		<classname>std::function</classname> parameter with a
		default do-nothing closure.
              </para>
            </listitem>

	    <listitem>
	      <para>
		Several C functions accept two parameters, a Unicode character
		pointer and the embedding level buffer, and a single parameter
		that specifies the size of both.
		The equivalent C++ function takes two discrete parameters,
		a <classname>std::u32string</classname> and a
		<classname>std::vector</classname> and returns an
		<classname>int</classname>; a negative value if their sizes
		differ, and 0 if their sizes match, and the requested function
		completes. The <function>unicode::bidi_embed</function> overload
		that returns a <classname>std::u32string</classname> returns
		an empty string in case of a mismatch.
              </para>
            </listitem>

	    <listitem>
	      <para>
		<function>unicode::bidi_reorder</function>
		reorders the entire <parameter>string</parameter> and its
		<parameter>embedding_level</parameter>s by default.
		The optional
		<parameter>starting_pos</parameter> and
		<parameter>n</parameter> parameters limit the reordering
		to the indicated subset of the original string (specified
		as the starting position offset index, and the number of
		characters).
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		<function>unicode::bidi_reorder</function>,
		<function>unicode::bidi_cleanup</function>,
		<function>unicode::bidi_logical_order</function>,
		<function>unicode::bidi_needs_embed</function> and
		<function>unicode::bidi_get_direction</function>
		take two optional
		parameters (defaulted values or overloaded) specifying
		an optional starting position and number of characters that
		define a subset of the original string that gets reordered,
		cleaned up, or has its direction determined.
	      </para>

	      <para>
		This <function>unicode::bidi_cleanup</function> does not
		trim off the passed in string and embedding level buffer,
		since it affects only a subset of the string. The number
		of times the removed character callback gets invoked
		indicates how much the substring should be trimmed off.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		<function>unicode::bidi_override</function>
		modifies the passed-in <parameter>string</parameter> as
		follows:
	      </para>

	      <itemizedlist>
		<listitem>
		  <para>
		    <function>unicode::bidi_cleanup</function>() is applied
		    with the specified, or defaulted,
		    <replaceable>cleanup_options</replaceable>
		  </para>
		</listitem>

		<listitem>
		  <para>
		    Either the <literal>LRO</literal> or an
		    <literal>RLO</literal> override marker gets prepended
		    to the Unicode string, forcing the entire string to
		    be interpreted in a single rendering direction, when
		    processed by the Unicode bi-directional algorithm.
		  </para>
		</listitem>
	      </itemizedlist>

	      <para>
		<function>unicode::bidi_override</function> makes it
		possible to use a Unicode-aware application or algorithm
		in a context that only works with text that's always
		displayed in a fixed direction, allowing graceful handling
		of input containing bi-directional text.
	      </para>
	    </listitem>
          </itemizedlist>

	  <refsect2 id="unicode_cpp_bidi_literals">
	    <title><literal>unicode::literals</literal> namespace</title>

	    <blockquote>
	      <informalexample>
		<programlisting><![CDATA[
using namespace unicode::literals;

std::u32string foo(std::u32string bar)
{
	return bar + LRO;
}
]]></programlisting>
	      </informalexample>
	    </blockquote>

	    <para>
	      This namespace contains the following <literal>constexpr</literal>
	      definitions:
	    </para>

	    <itemizedlist>
	      <listitem>
		<para>
		  <classname>char32_t</classname> arrays with literal
		  Unicode character strings containing Unicode directional,
		  isolate, and override markers, like
		  <literal>LRO</literal>,
		  <literal>RLO</literal> and others.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  <literal>CLEANUP_EXTRA</literal>,
		  <literal>CLEANUP_BNL</literal>, and
		  <literal>CLEANUP_CANONICAL</literal> options for
		  <function>unicode::bidi_cleanup</function>().
		</para>
	      </listitem>
	    </itemizedlist>

	  </refsect2>
	</refsect1>

	<refsect1 id="unicode_cpp_bidi_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_bidi">
	      <citerefentry><refentrytitle>unicode_bidi</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
          </para>
        </refsect1>
      </refentry>

      <refentry id="unicode__canonical">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::canonical</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::canonical</refname>
	  <refname>unicode::decompose</refname>
	  <refname>unicode::decompose_default_reallocate</refname>
	  <refname>unicode::compose</refname>
	  <refname>unicode::compose_default_callback</refname>

	  <refpurpose>unicode canonical normalization and denormalization</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;

constexpr int decompose_flag_qc=UNICODE_DECOMPOSE_FLAG_QC;
constexpr int decompose_flag_compat=UNICODE_DECOMPOSE_FLAG_COMPAT;

constexpr int compose_flag_removeunused=UNICODE_COMPOSE_FLAG_REMOVEUNUSED;
constexpr int compose_flag_oneshot=UNICODE_COMPOSE_FLAG_ONESHOT;</funcsynopsisinfo>

	    <funcprototype>
	      <funcdef>void <funcname>decompose_default_reallocate</funcname></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::vector&lt;std::tuple&lt;size_t, size_t&gt;&gt; &amp;<parameter>list</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <funcname>decompose</funcname></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>int <parameter>flags</parameter>=0</paramdef>
	      <paramdef>const std::function&lt;void (std::u32string &amp;, const std::vector&lt;std::tuple&lt;size_t, size_t&gt;&gt;)&gt; &amp;<parameter>reallocate</parameter>=decompose_default_reallocate</paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <funcname>compose_default_callback</funcname></funcdef>
	      <paramdef>unicode_composition_t &amp;<parameter>compositions</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
	      <funcdef>void <funcname>compose</funcname></funcdef>
	      <paramdef>std::u32string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>int <parameter>flags</parameter>=0</paramdef>
	      <paramdef>const std::function&lt;void (unicode_composition_t &amp;)&gt; &amp;<parameter>cb</parameter>=compose_default_reallocate</paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_canonical_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions implement the C++ interface for the
	    <ulink url="https://www.unicode.org/reports/tr15/tr15-&tr15ver;.html">Unicode Canonical Decomposition and Composition</ulink>,
	    See the description of the underlying
	    <link linkend="unicode_canonical">
	      <citerefentry><refentrytitle>unicode_canonical</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link> C library
	      API for more information. C++ specific notes:
	  </para>

	  <itemizedlist>
	    <listitem>
	      <para>
		The C++ decomposition <parameter>reallocate</parameter> callback
		receives a single vector of <replaceable>offset</replaceable>
		and
		<replaceable>size</replaceable> tuples instead of two separate
		arrays or vectors.
		<function>unicode::decompose_default_reallocate</function>() is
		the C++ version of the default
		<varname>reallocate</varname> callback. It receives the
		receiving the same tuple vector parameter, too.
		The C++ interface use <classname>std::u32string</classname>s
		to represent Unicode text strings, and
		<function>unicode::decompose_default_reallocate</function>()
		<function>resize</function>s it.
	      </para>

	      <para>
		Like the C callback, the C++ one gets called 0 or more times.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		<function>unicode::compose</function>() takes care of
		initializing, applying, and de-initialization
		the <classname>unicode_composition_t</classname> object,
		for decomposition.
		The callback receives a reference to the
		<classname>unicode_composition_t</classname> object, which
		the callback should not modify in any way.
	      </para>
	    </listitem>
	  </itemizedlist>
	</refsect1>
	<refsect1 id="unicode_cpp_canonical_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_canonical">
	      <citerefentry>
		<refentrytitle>unicode_canonical</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>
      <refentry id="unicode__iconvert__convert">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::iconvert::convert</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::iconvert::convert</refname>
	  <refname>unicode::ucs_4</refname>
	  <refname>unicode::ucs_2</refname>
	  <refname>unicode::utf_8</refname>
	  <refname>unicode::iso_8859_1</refname>

	  <refpurpose>unicode character set conversion</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;

extern const char unicode::ucs_4[];
extern const char unicode::ucs_2[];
extern const char unicode::utf_8[];
extern const char unicode::iso_8859_1[];</funcsynopsisinfo>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>srccharset</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>dstcharset</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>srccharset</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>dstcharset</parameter></paramdef>
              <paramdef>bool &amp;<parameter>errflag</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert</function></funcdef>
              <paramdef>const std::vector&lt;char32_t&gt; &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>dstcharset</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert</function></funcdef>
              <paramdef>const std::vector&lt;char32_t&gt; &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>dstcharset</parameter></paramdef>
              <paramdef>bool &amp;<parameter>errflag</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>bool <function>unicode::iconvert::convert</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>std::vector&lt;char32_t&gt; &amp;<parameter>text</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_convert_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    The overloaded
	    <function>unicode::convert::convert</function>() functions
	    convert:
	  </para>

	  <itemizedlist>
	    <listitem>
	      <para>
		A text string between two different character sets, returning
		the new string.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		A vector of unicode characters (not null-terminated) to
		a character string in a supported character set.
	      </para>
	    </listitem>

	    <listitem>
	      <para>
		Initialize a vector of unicode characters, passed by
		reference, by converting a text string in a given character
		set to unicode.
	      </para>
	    </listitem>
	  </itemizedlist>

	  <para>
	    These functions use
	    <citerefentry><refentrytitle>iconv</refentrytitle>
	    <manvolnum>3</manvolnum></citerefentry>, and can use any
	    character set that's supported by
	    <citerefentry><refentrytitle>iconv</refentrytitle>
	    <manvolnum>3</manvolnum></citerefentry>.

	    Use
	    <varname>unicode::ucs_2</varname> and
	    <varname>unicode::ucs_4</varname> to specify the 16 and the 32 bit
	    unicode octet in native byte order.
	    Use
	    <varname>unicode::utf_8</varname> and
	    <varname>unicode::iso_8859_1</varname> to specify these two
	    standard character sets.

	    The overloaded versions that pass a reference to a
	    <classname>bool</classname> set the flag to <literal>true</literal>
	    if some characters could not be converted.
	    The overloaded version that initializes a unicode vector returns
	    the <classname>bool</classname> flag, instead.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_convert_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert_tocase">
	      <citerefentry><refentrytitle>unicode::convert::convert_tocase</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry><refentrytitle>unicode_convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></ulink>.

	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__iconvert__convert_tocase">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::iconvert::convert_tocase</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::iconvert::convert_tocase</refname>

	  <refpurpose>unicode uppercase, lowercase, and titlecase conversion</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert_tocase</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>char32_t (*<parameter>first_char_func</parameter>)(char32_t)</paramdef>
              <paramdef>char32_t (*<parameter>char_func</parameter>)(char32_t)</paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::iconvert::convert_tocase</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>bool &amp;<parameter>err</parameter></paramdef>
              <paramdef>char32_t (*<parameter>first_char_func</parameter>)(char32_t)</paramdef>
              <paramdef>char32_t (*<parameter>char_func</parameter>)(char32_t)</paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_convert_tocase_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    The overloaded
	    <function>unicode::convert::convert_tocase</function>() function
	    converts the <parameter>text</parameter> parameter, in the
	    <parameter>charset</parameter> characters to lowercase, uppercase,
	    and titlecase. <parameter>text</parameter> gets converted,
	    internally, into unicode.
            <parameter>first_char_func</parameter> and
            <parameter>char_func</parameter> are either:
	    <function>unicode_lc</function>,
	    <function>unicode_uc</function>, or
	    <function>unicode_tc</function>. If the converted text string is
	    not empty, <parameter>first_char_func</parameter> converts the
	    first unicode character in the text string, and
            <parameter>char_func</parameter> converts any remaining characters.
	    <function>unicode_lc</function> converts its character to lowercase,
	    <function>unicode_uc</function> to uppercase, and
	    <function>unicode_tc</function> to titlecase. Finally, the
	    unicode string gets converted back to
	    <parameter>charset</parameter>, which gets returned.
	  </para>

	  <para>
	    The optional <parameter>err</parameter> parameter gets set to true if
	    an error was encounted converting the text string to or from
	    unicode.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_convert_tocase_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert">
	      <citerefentry><refentrytitle>unicode::convert::convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry><refentrytitle>unicode_convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></ulink>.

	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__iconvert__fromu">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::iconvert::fromu</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::iconvert::fromu</refname>

	  <refpurpose>template for converting text sequence from unicode</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>

	    <funcprototype>
              <funcdef>output_iter_t <function>unicode::iconvert::fromu::convert</function></funcdef>
              <paramdef>input_iter_t <parameter>beg_iter</parameter></paramdef>
              <paramdef>input_iter_t <parameter>end_iter</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>output_iter_t <parameter>output_iter</parameter></paramdef>
              <paramdef>bool &amp;<parameter>errflag</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>void <function>unicode::iconvert::fromu::convert</function></funcdef>
              <paramdef>input_iter_t <parameter>beg_iter</parameter></paramdef>
              <paramdef>input_iter_t <parameter>end_iter</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>std::string &amp;<parameter>out_buf</parameter></paramdef>
              <paramdef>bool &amp;<parameter>errflag</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::pair&lt;std::string, bool&gt; <function>unicode::iconvert::fromu::convert</function></funcdef>
              <paramdef>const std::u32string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_convert_fromu_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These template functions convert unicode characters to
	    text in the given character set.
	    <parameter>beg_iter</parameter> and
	    <parameter>end_iter</parameter> define an input sequence of
	    <classname>char32_t</classname>s.
	    They get converted to unicode characters.
	    <parameter>output_iter</parameter> is an output iterator that
	    <function>convert</function>()
	    iterates over <classname>char</classname>s in the specified
	    character set.
	    <function>convert</function>() returns the value of the output
	    iterator after iterating over the converted character sequence.
	    <parameter>err_flag</parameter> gets set to <literal>true</literal>
	    if unicode text could not be converted to the requested character
	    set, or <literal>false</literal> for a successful conversion.
	  </para>

	  <para>
	    An overloaded <function>convert</function>() puts the text string
	    into a <classname>std::string</classname>, instead of using
	    an output iterator.
	    Finally, a single
	    <classname>std::u32string</classname>
	    specifies the character string, instead of a beginning and an
	    ending iterator.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_convert_fromu_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert">
	      <citerefentry><refentrytitle>unicode::convert::convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry><refentrytitle>unicode_convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></ulink>.

	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__iconvert__tou">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::iconvert::tou</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::iconvert::tou</refname>

	  <refpurpose>template for converting text sequence to unicode</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>

	    <funcprototype>
              <funcdef>output_iter_t <function>convert</function></funcdef>
              <paramdef>input_iter_t <parameter>beg_iter</parameter></paramdef>
              <paramdef>input_iter_t <parameter>end_iter</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
	      <paramdef>bool &amp;<parameter>errflag</parameter></paramdef>
              <paramdef>output_iter_t <parameter>output_iter</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>bool <function>convert</function></funcdef>
              <paramdef>input_iter_t <parameter>beg_iter</parameter></paramdef>
              <paramdef>input_iter_t <parameter>end_iter</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
              <paramdef>std::u32string &amp;<parameter>out_buf</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::pair&lt;std::u32string, bool&gt; <function>convert</function></funcdef>
              <paramdef>const std::string &amp;<parameter>text</parameter></paramdef>
              <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_convert_tou_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These template functions convert text in a given character set
	    to unicode characters.
	    <parameter>beg_iter</parameter> and
	    <parameter>end_iter</parameter> define an input sequence of
	    <classname>char</classname>s in the <parameter>charset</parameter>
	    character set. They get converted to unicode characters.
	    <parameter>output_iter</parameter> is an output iterator that
	    <function>convert</function>()
	    iterates over <classname>char32_t</classname>s.
	    <function>convert</function>() returns the value of the output
	    iterator after iterating over the converted character sequence.
	    <parameter>errflag</parameter>, passed by reference, gets set to
	    <literal>true</literal> if some character could not be converted
	    to unicode, from the specified character set, and
	    <literal>false</literal> if the conversion completed without
	    errors.
	  </para>

	  <para>
	    An overloaded <function>convert</function>() puts the unicode
	    character sequence into a vector of
	    <classname>char32_t</classname>s, instead of an output
	    sequence, and returned the error flag.
	    Finally, a single <classname>std::string</classname>
	    specifies the character string, instead of a beginning and an
	    ending iterator, and returns a
	    <classname>std::pair</classname> with the converted unicode
	    text in a vector, and the error flag.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_convert_tou_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode__iconvert__convert">
	      <citerefentry><refentrytitle>unicode::convert::convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	    <link linkend="unicode_convert">
	      <citerefentry><refentrytitle>unicode_convert</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      <ulink url="https://manpages.courier-mta.org/htmlman3/iconv.3.html">
	      <citerefentry><refentrytitle>iconv</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></ulink>.

	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__linebreak">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::linebreak</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::linebreak_callback_base</refname>
	  <refname>unicode::linebreak_callback_save_buf</refname>
	  <refname>unicode::linebreakc_callback_base</refname>
	  <refname>unicode::linebreak_iter</refname>
	  <refname>unicode::linebreakc_iter</refname>

	  <refpurpose>unicode line-breaking rules</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <programlisting>
#include &lt;courier-unicode.h&gt;

class linebreak : public unicode::linebreak_callback_base {

public:

    using unicode::linebreak_callback_base::operator&lt;&lt;;
    using unicode::linebreak_callback_base::operator();
    int callback(int linebreak_code)
    {
        // ...
    }
};

char32_t c;
std::u32string buf;

linebreak compute_linebreak;

compute_linebreak.set_opts(UNICODE_LB_OPT_SYBREAK);
compute_linebreak &lt;&lt; c;

compute_linebreak(buf);
compute_linebreak(buf.begin(), buf.end());

compute_linebreak.finish();

// ...

unicode::linebreak_callback_save_buf linebreaks;

std::list&lt;int&gt; lb=linebreaks.lb_buf;

class linebreakc : public unicode::linebreakc_callback_base {

public:

    using unicode::linebreak_callback_base::operator&lt;&lt;;
    using unicode::linebreak_callback_base::operator();
    int callback(int linebreak_code, char32_t ch)
    {
        // ...
    }
};

// ...

std::u32string buf;

typedef unicode::linebreak_iter&lt;std::u32string::const_iterator&gt; iter_t;

iter_t beg_iter(buf.begin(), buf.end()), end_iter;

beg_iter.set_opts(UNICODE_LB_OPT_SYBREAK);

std::vector&lt;int&gt; linebreaks;

std::copy(beg_iter, end_iter, std::back_insert_iterator&lt;std::vector&lt;int&gt;&gt;(linebreaks));

// ...

typedef unicode::linebreakc_iter&lt;std::u32string::const_iterator&gt; iter_t;

iter_t beg_iter(buf.begin(), buf.end()), end_iter;

beg_iter.set_opts(UNICODE_LB_OPT_SYBREAK);

std::vector&lt;std::pair&lt;int, char32_t&gt;&gt; linebreaks;

std::copy(beg_iter, end_iter, std::back_insert_iterator&lt;std::vector&lt;int&gt;&gt;(linebreaks));</programlisting>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_lb_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    <classname>unicode::linebreak_callback_base</classname> is a C++
	    binding for the unicode line-breaking rule implementation described
	    in
	    <link linkend="unicode_line_break">
	      <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>

	  <para>
	    Subclass <classname>unicode::linebreak_callback_base</classname>
	    and implement <methodname>callback</methodname>() that's virtually
	    inherited from
	    <classname>unicode::linebreak_callback_base</classname>.
	    The
	    <methodname>callback</methodname>() callback function receives the
	    output values from the line-breaking algorithm, the
	    <literal>UNICODE_LB_MANDATORY</literal>,
	    <literal>UNICODE_LB_NONE</literal>, or the
	    <literal>UNICODE_LB_ALLOWED</literal> value, for each unicode
	    character.
	  </para>

	  <para>
	    <methodname>callback</methodname>() should return 0. A non-zero
	    return reports an error, that stops the line-breaking algorithm.
	    See
	    <link linkend="unicode_line_break">
	      <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link> for more
	      information.
	  </para>

	  <para>
	    The alternate
	    <classname>unicode::linebreakc_callback_base</classname>
	    interface uses a virtually inherited
	    <methodname>callback</methodname>() that receives two parameters,
	    the line-break code value, and the corresponding unicode character.
	  </para>
 	  <para>
	    The input unicode characters for the line-breaking
	    algorithm are provided by the <literal>&lt;&lt;</literal>
	    operator, one unicode character at a time; or by the
	    <literal>()</literal> operator, passing either a container, or
	    a beginning and an ending iterator value for an input sequence
	    of unicode characters. <methodname>finish</methodname>() indicates
	    the end of the unicode character sequence.
	  </para>

	  <para>
	    <methodname>set_opts</methodname> sets line-breaking options
	    (see <methodname>unicode_lb_set_opts</methodname>() for more
	    information).
	  </para>
	  <para>
	    <classname>unicode::linebreak_callback_save_buf</classname> is a
	    subclass that implements <methodname>callback</methodname>() by
	    saving the linebreaks codes into a <classname>std::list</classname>.
	  </para>

	  <para>
	    The <classname>linebreak_iter</classname> template implements an
	    input iterator over <classname>int</classname>s.
	    The template parameter is an input iterator over
	    <classname>unicode</classname> chars. The constructor's parameters
	    are a beginning and an ending iterator value for a sequence of
	    <classname>char32_t</classname>. This constructs the beginning
	    iterator value for a sequence of <classname>int</classname>s
	    consisting of line-break values
	    (<literal>UNICODE_LB_MANDATORY</literal>,
	    <literal>UNICODE_LB_NONE</literal>, or
	    <literal>UNICODE_LB_ALLOWED</literal>) corresponding to each
	    <classname>char32_t</classname> in the underlying sequence.
	    The default constructor creates the ending iterator value for the
	    sequence.
	  </para>

	  <para>
	    The iterator implements a <methodname>set_opts</methodname>()
	    methods that sets the options for the line-breaking algorithm.
	  </para>

	  <para>
	    The <classname>linebreakc_iter</classname> template implements a
	    similar input iterator, with the difference that it ends up iterating
	    over a <classname>std::pair</classname> of line-breaking values and
	    the corresponding <classname>char32_t</classname> from the
	    underlying input sequence.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_lb_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_line_break">
	      <citerefentry><refentrytitle>unicode_line_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__tolower">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::tolower</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::tolower</refname>
	  <refname>unicode::toupper</refname>
	  <refpurpose>unicode version of
	  <citerefentry><refentrytitle>tolower</refentrytitle>
	  <manvolnum>3</manvolnum></citerefentry>
	  and
	  <citerefentry><refentrytitle>toupper</refentrytitle>
	  <manvolnum>3</manvolnum></citerefentry>
	  </refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <funcsynopsis>
	    <funcsynopsisinfo>#include &lt;courier-unicode.h&gt;</funcsynopsisinfo>
	    <funcprototype>
              <funcdef>std::string <function>unicode::tolower</function></funcdef>
	      <paramdef>const std::string &amp;<parameter>string</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::tolower</function></funcdef>
	      <paramdef>const std::string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::u32string <function>unicode::tolower</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>u</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::toupper</function></funcdef>
	      <paramdef>const std::string &amp;<parameter>string</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::string <function>unicode::toupper</function></funcdef>
	      <paramdef>const std::string &amp;<parameter>string</parameter></paramdef>
	      <paramdef>const std::string &amp;<parameter>charset</parameter></paramdef>
	    </funcprototype>

	    <funcprototype>
              <funcdef>std::u32string <function>unicode::toupper</function></funcdef>
	      <paramdef>const std::u32string &amp;<parameter>u</parameter></paramdef>
	    </funcprototype>
	  </funcsynopsis>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_tolower_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    These functions convert the <replaceable>string</replaceable>
	    parameter, in <replaceable>charset</replaceable> or
	    <link linkend="unicode_default_chset">
	      <citerefentry><refentrytitle>unicode_default_chset</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>,
	      to unicode, replace each character with
	      <link linkend="unicode_uc">
		<citerefentry><refentrytitle>unicode_lc</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link> or
	      <link linkend="unicode_uc">
		<citerefentry><refentrytitle>unicode_uc</refentrytitle>
		<manvolnum>3</manvolnum></citerefentry></link>,
		then convert it back to the same character set, returning
		the resulting string.
	  </para>

	  <para>
	    Passing a
	    <classname>const std::u32string &amp;</classname>
	    directly also converts it accordingly, returning the converted
	    unicode string.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_tolower_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

      <refentry id="unicode__wordbreak">
	<refentryinfo><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Unicode Library</productname></refentryinfo>

	<refmeta>
	  <refentrytitle>unicode::wordbreak</refentrytitle>
	  <manvolnum>3</manvolnum>
	</refmeta>

	<refnamediv>
	  <refname>unicode::wordbreak_callback_base</refname>
	  <refname>unicode::wordbreak</refname>
	  <refpurpose>unicode word-breaking rules</refpurpose>
	</refnamediv>

	<refsynopsisdiv>
	  <programlisting>
#include &lt;courier-unicode.h&gt;

class wordbreak : public unicode::wordbreak_callback_base {

public:

    using unicode::wordbreak_callback_base::operator&lt;&lt;;
    using unicode::wordbreak_callback_base::operator();
    int callback(bool flag)
    {
        // ...
    }
};

char32_t c;
std::u32string buf;

wordbreak compute_wordbreak;

compute_wordbreak &lt;&lt; c;

compute_wordbreak(buf);
compute_wordbreak(buf.begin(), buf.end());

compute_wordbreak.finish();

// ...

unicode_wordbreakscan scan;

scan &lt;&lt; c;

size_t nchars=scan.finish();

</programlisting>
	</refsynopsisdiv>

	<refsect1 id="unicode_cpp_wb_descr">
	  <title>DESCRIPTION</title>

	  <para>
	    <classname>unicode::wordbreak_callback_base</classname> is a C++
	    binding for the unicode word-breaking rule implementation described
	    in
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>

	  <para>
	    Subclass <classname>unicode::wordbreak_callback_base</classname>
	    and implement <methodname>callback</methodname>() that's virtually
	    inherited from
	    <classname>unicode::wordbreak_callback_base</classname>.
	    The
	    <methodname>callback</methodname>() callback function receives the
	    output values from the word-breaking algorithm, namely a
	    <classname>bool</classname> indicating whether a word break
	    exists before the unicode character in the underlying input sequence.
	  </para>

	  <para>
	    <methodname>callback</methodname>() should return 0. A non-zero
	    return reports an error, that stops the word-breaking algorithm.
	    See
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link> for more
	      information.
	  </para>

 	  <para>
	    The input unicode characters for the word-breaking
	    algorithm are provided by the <literal>&lt;&lt;</literal>
	    operator, one unicode character at a time; or by the
	    <literal>()</literal> operator, passing either a container, or
	    a beginning and an ending iterator value for an input sequence
	    of unicode characters. <methodname>finish</methodname>() indicates
	    the end of the unicode character sequence.
	  </para>

	  <para>
	    <classname>unicode::wordbreakscan</classname> is a C++
	    binding for the
	    <function>unicode_wbscan_init</function>(),
            <function>unicode_wbscan_next</function>() and
            <function>unicode_wbscan_end</function>
	    methods described in
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	      Its <methodname>&lt;&lt;</methodname> iterates over the
	      unicode characters, and <methodname>finish</methodname>()
	      indicates the number of characters before the first unicode
	      word break. The <methodname>&lt;&lt;</methodname> iterator
	      returns a <classname>bool</classname> indicating when the first
	      word break has already been found, so further calls are not
	      necessary.
	  </para>
	</refsect1>

	<refsect1 id="unicode_cpp_wb_seealso">
	  <title>SEE ALSO</title>
	  <para>
	    <link linkend="courier-unicode">
	      <citerefentry>
		<refentrytitle>courier-unicode</refentrytitle>
		<manvolnum>7</manvolnum></citerefentry></link>,
	    <link linkend="unicode_word_break">
	      <citerefentry><refentrytitle>unicode_word_break</refentrytitle>
	      <manvolnum>3</manvolnum></citerefentry></link>.
	  </para>
	</refsect1>
      </refentry>

    </section>
  </section>
  <section id="COPYING">
    <title>COPYING</title>

    <para role="COPYING">
      The Courier Unicode Library is free software, distributed under the
      terms of the GPL, version 3:
    </para>
    <blockquote>
      <literallayout><xi:include href="COPYING" parse="text" xmlns:xi="http://www.w3.org/2001/XInclude" /></literallayout>
    </blockquote>
  </section>
</article>