path: root/maildir/maildirmake.sgml

<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<!-- Copyright 1998 - 2018 Double Precision, Inc.  See COPYING for -->
<!-- distribution information. -->
<refentry id="maildirmake">
  <info><author><firstname>Sam</firstname><surname>Varshavchik</surname><contrib>Author</contrib></author><productname>Courier Mail Server</productname></info>

  <refmeta>
    <refentrytitle>maildirmake</refentrytitle>
    <manvolnum>1</manvolnum>
    <refmiscinfo class="manual">Double Precision, Inc.</refmiscinfo>
  </refmeta>

  <refnamediv>
    <refname>maildirmake</refname>
    <refpurpose>create maildirs and maildir folders</refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <cmdsynopsis sepchar=" ">
      <command>maildirmake</command>
      <arg choice="opt" rep="repeat">options</arg>
      <arg choice="req" rep="norepeat"><replaceable>maildir</replaceable></arg>
    </cmdsynopsis>
  </refsynopsisdiv>

  <refsect1 id="description">
    <title>DESCRIPTION</title>

    <para>
The <command>maildirmake</command> command creates maildirs, and
maildir folders and performs some routine maintenance on them.
This documentation
describes the <command>maildirmake</command> command from the
<application>Courier</application> mail server,
which creates an extended form of maildirs that implements
additional extensions beyond the basic maildir properties that were first
implemented in the Qmail mail server.</para>
  </refsect1>
  <refsect1 id="options">
      <title>OPTIONS</title>

    <variablelist>
      <varlistentry><term><literal>-S</literal></term>
        <listitem><para>create a "sharable" maildir.  A sharable maildir has
slightly different permissions which allows creation of publicly-shared
folders.</para>
	</listitem>
      </varlistentry>

      <varlistentry><term><literal>-q</literal> <replaceable>quota</replaceable></term>
        <listitem>
	  <para>install a quota on the maildir.
	    See
	    <ulink url="maildirquota.html"><citerefentry><refentrytitle>maildirquota</refentrytitle><manvolnum>7</manvolnum></citerefentry></ulink>
	    for more information.
	    The specified maildir gets automatically created if it does not
	    exist; otherwise the existing maildir's quota gets updated.
	    <replaceable>quota</replaceable> may be:
	  </para>
	  <variablelist>
	    <varlistentry>
	      <term><replaceable>####S</replaceable></term>
	      <listitem>
		<para>
		  Specifies the quota in the total number of bytes for all
		  messages.
		  <quote>10000000S</quote> specifies a quota of ten million
		  bytes.
		</para>
	      </listitem>
	    </varlistentry>
	    <varlistentry>
	      <term><replaceable>####C</replaceable></term>
	      <listitem>
		<para>
		  Specifies the quota in the total number of messages in
		  the maildir.
		  <quote>10000S</quote> specifies a quota of ten thousand
		  messages.
		</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	</listitem>
      </varlistentry>

      <varlistentry><term><literal>-f</literal> <replaceable>folder</replaceable></term>
        <listitem><para>do not create a maildir, but create a folder in an
existing maildir.</para>
	</listitem>
      </varlistentry>

      <varlistentry><term><literal>-F</literal> <replaceable>folder</replaceable></term>
        <listitem><para>Like the <literal>-f</literal> option, except
	    that the folder's name is given using the system locale's
	    character set. Non-Latin characters in the folder's name
	    must be given to the <literal>-f</literal> option using IMAP's
	    UTF8 encoding. The <literal>-F</literal> option
	    takes the folder name specified using the console's character
	    set.</para>
	</listitem>
      </varlistentry>

      <varlistentry><term><literal>-s</literal> <replaceable>mode</replaceable></term>
        <listitem><para>create a publicly accessible folder in an
existing sharable maildir.  First, use the <option>-S</option> option to
create a sharable maildir.
Then, run <command>maildirmake</command> again with the
<option>-s</option> option to create
publicly accessible folders.
<replaceable>mode</replaceable> is a comma-separated list of
the following keywords: <literal>read</literal> - readonly folder, only you can
write messages to this folder;
<literal>write</literal> - anyone can read and
write messages to this folder;
<literal>group</literal> - only allow members of
your own system group to access messages in this folder (instead of
everyone).</para>
	</listitem>
      </varlistentry>

      <varlistentry><term><literal>--add</literal> <replaceable>name</replaceable>=<replaceable>pathname</replaceable>,
<literal>--del</literal> <replaceable>name</replaceable></term>
        <listitem><para>
create or delete the directories and links needed to
access shared folders.  See below for more information.</para>
	</listitem>
      </varlistentry>
      <varlistentry>
	<term><literal>--checkutf8</literal> <replaceable>maildir</replaceable> <replaceable>maildirfilter</replaceable></term>
	<listitem>
	  <para>
	    Perform a sanity check to verify that a pre-unicode format
	    maildir can be converted to a unicode-format maildir.
	    See <quote>Converting pre-unicode format maildirs</quote>, below,
	    for more information.
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><literal>--convutf8</literal> <replaceable>maildir</replaceable> <replaceable>maildirfilter</replaceable></term>
	<listitem>
	  <para>
	    Convert a pre-unicode format
	    maildir can be converted to a unicode-format maildir.
	    See <quote>Converting pre-unicode format maildirs</quote>, below,
	    for more information.
	  </para>
	</listitem>
      </varlistentry>
    </variablelist>

    <refsect2 id="folders">
      <title>FOLDERS</title>

      <para>
This <command>maildirmake</command> command supports enhanced maildirs that
contain folders.</para>

      <para>
By itself, <command>maildirmake</command> makes a new subdirectory
<replaceable>maildir</replaceable>,
and creates all the necessary structures.
The <option>-f</option> option
creates a new "folder" within an existing
<replaceable>maildir</replaceable>. <replaceable>maildir</replaceable> must
already exist, and the <command>maildirmake</command> command will create a new
folder in the maildir.</para>

      <para>
Folders are simply subdirectories inside the main maildir whose names start
with a period, and which are themselves maildirs.
For example, the command
"<command>maildirmake -f Drafts mail/Maildir</command>" creates
<filename>mail/Maildir/.Drafts</filename>,
that has the usual <filename>tmp</filename>,
<filename>new</filename> and <filename>cur</filename>.
You MUST use the <option>-f</option> option, instead of
specifying <filename>mail/Maildir/.Drafts</filename> directly,
in order to correctly
initialize certain enhanced maildir features.</para>

      <para>Folders cannot be created directly within other folders.
Running
<command>maildirmake -f Urgent mail/Maildir/.Drafts</command> will not work.
Instead, the period character is designated as a hierarchy separator, run
<command>maildirmake -f Drafts.Urgent mail/Maildir</command> instead.
This creates
<filename>mail/Maildir/.Drafts.Urgent</filename>, and all mail software
that supports
enhanced maildirs will interpret it as a subfolder Urgent of the Drafts
folder.</para>

    </refsect2>

    <refsect2 id="sharedfolders">
      <title>SHARED FOLDERS</title>

      <para>
This is another extension to the Maildir format that allows folders to be
shared between multiple clients.</para>

      <note>
	<para>
The <application>Courier</application>
IMAP server implements two types of shared folders:
filesystem permission-based shared folders,
as well as virtual shared folders based on IMAP access control lists.
Use the <command>maildirmake</command> command
to implement shared folders based on
filesystem permissions.
The
<ulink url="maildiracl.html"><citerefentry><refentrytitle>maildiracl</refentrytitle><manvolnum>1</manvolnum></citerefentry></ulink>
command manages access control lists, which
are used by virtual shared folders.</para>

	<para>
See the <application>Courier</application>
IMAP server documentation for more information.</para>
      </note>

      <para>
First, you need to create a collection of
sharable folders, as a separate maildir:</para>
      <blockquote>
	<informalexample>
	  <literallayout format="linespecific" class="normal"><command>maildirmake -S /usr/local/share/maildirs/notices</command></literallayout>
	</informalexample>
      </blockquote>

      <para>
Then, create individuals folders that will be accessed in shared mode:</para>

      <blockquote>
	<informalexample>
	  <literallayout format="linespecific" class="normal"><command>maildirmake -s write -f Weekly /usr/local/share/maildirs/notices</command></literallayout>
	</informalexample>
      </blockquote>

      <para>In this example, the "Weekly" folder is created,
with read/write access to everyone.
Multiple folders can be created in the same maildir, with different access
permissions. Everyone can create a sharable maildir.  The access privileges
for individual folders are set by the <option>-s</option> option, and are
implemented using traditional filesystem permissions.</para>

    <para>Use the <option>--add</option> and
<option>--del</option> options to add a sharable maildir to
an existing maildir.  Client software that implements this extension will now
know where to find sharable folders:</para>
    <blockquote>
      <informalexample>
	<literallayout format="linespecific" class="normal"><command>maildirmake --add notices=/usr/local/share/maildirs/notices $HOME/Maildir</command></literallayout>
      </informalexample>
    </blockquote>

      <para>
<filename>$HOME/Maildir</filename> is your main maildir.
The argument to <option>-add</option>
is <replaceable>nick</replaceable>=<replaceable>path</replaceable>.
<replaceable>nick</replaceable> is a nickname for this collection of
sharable folders, and <replaceable>path</replaceable> is the location of the
sharable maildir.
All folders in the sharable maildir that you have access to -- such
as "Weekly", in this case, will now be accessible.
Multiple sharable maildirs can be added, by giving each one a unique
<replaceable>nick</replaceable>.</para>

      <para>
The <option>--del</option> option "disconnects" the sharable maildir from
the main maildir.</para>
    </refsect2>

    <refsect2 id="globalsharedfolders">
      <title>GLOBAL SHARED FOLDERS</title>

      <para>
Normally <option>-add</option> command must be run for every maildir
which needs
to access the sharable maildir. Alternatively the file
<filename>@sysconfdir@/maildirshared</filename> can be created,
to specify a default set of sharable maildirs.
Each line in this file takes the following format:</para>
      <blockquote>
	<informalexample>
	  <literallayout format="linespecific" class="normal"><replaceable>nick</replaceable><token>&lt;tab&gt;</token><replaceable>path</replaceable></literallayout>
	</informalexample>
      </blockquote>

      <para><replaceable>nick</replaceable> is a short nickname for
the sharable maildir, <token>&lt;tab&gt;</token>
is a single tab character, <replaceable>path</replaceable>
is the pathname to the sharable maildir.</para>
    </refsect2>

    <refsect2 id="sharedaccess">
      <title>ACCESSING SHARED FOLDERS</title>

      <para>
You may have read or write access to a shared folder.
If you have write
access, you can add messages to the shared folder. You can also delete
messages that you've added.</para>

      <para>Anyone can create a sharable maildir, so if the sharable maildir
is actually created by you, can can delete any message, not just your
own.</para>
    </refsect2>

  </refsect1>

  <refsect1 id="converting">
    <title>Converting pre-unicode format maildirs</title>

    <para>
      This section is relevant to:

    </para>

    <itemizedlist>
      <listitem>
	<para>
	  Updating <application>Courier-IMAP</application> to version 5.0, and
	  later, from prior versions of <application>Courier-IMAP</application>, or:
	</para>
      </listitem>
      <listitem>
	<para>
	  Updating <application>SqWebmail</application> to version 6.0, and
	  later, from prior versions of <application>SqWebmail</application>, or:
	</para>
      </listitem>
      <listitem>
	<para>
	  Updating Courier to version 1.0, and
	  later, from prior versions of Courier.
	</para>
      </listitem>
    </itemizedlist>

    <para>
      These versions have been updated to implement native Unicode
      support in several E-mail-related protocols. It is already expected
      that updating Internet standards to use native Unicode-formatted
      E-mail messages will not be 100% backwards-compatible, in terms of
      E-mail client support.
      Given that, this major update to Unicode will also introduce some
      backwards-incompatible changes to the internal structure of maildirs,
      as a major upgrade to simplify Unicode support going forward.
      Might as well go through the pain of a major upgrade once.
    </para>

    <para>
      <command>maildirmake</command>'s <option>--checkutf8</option> and
      <option>--convutf8</option> options are tools to aid in conversion of
      existing mailboxes to the new Unicode-based naming standard.
    </para>

    <refsect2 id="background">
      <title>Background</title>

      <para>
	Mail folders in a maildir are hidden subdirectories. For example:
	a folder name <quote>Mailing list</quote> is a maildir subdirectory
	named <filename>$HOME/Maildir/.Mailing list</filename>
	(<filename>$HOME/Maildir</filename> is the main mailbox).
      </para>

      <para>
	Prior to the unicode update, non-English characters in folder names
	used a convention based on the
	non-standard <quote>modified-UTF7</quote> encoding used by IMAP.
	A folder named <quote>Résumé</quote> is a maildir subdirectory
	named <filename>$HOME/Maildir/.R&amp;AOk-sum&amp;AOk-</filename>.
	The current versions of Courier,
	<application>Courier-IMAP</application>, and SqWebmail,
	now creates <filename>$HOME/Maildir/.Résumé</filename>
	using the <acronym>UTF8</acronym> encoding. This appears as plain
	<quote>.Résumé</quote> (hidden) subdirectory on modern
	UTF8-based systems.
      </para>

      <para>
	Consequently, any existing maildirs with folders that use non-English
	names
	must be converted as part of updating to the current version of
	Courier, <application>Courier-IMAP</application>, and SqWebmail from pre-unicode versions.
	This does not happen automatically when updating to the current
	version. This must be done manually given the wide variety of individual
	mail server configurations that are possible.
      </para>
    </refsect2>

    <refsect2 id="unicode">
      <title>Unicode conversion overview</title>

      <para>
	Updating from pre-unicode versions involves:
      </para>

      <itemizedlist>
	<listitem>
	  <para>
	    Renaming the actual maildir folders,
	    <filename>$HOME/Maildir/.<replaceable>names</replaceable></filename>
	    into unicode names (using <acronym>UTF8</acronym>).
	  </para>
	</listitem>

	<listitem>
	  <para>
	    Updating the
	    <filename>$HOME/Maildir/courierimapsubscribed</filename> file,
	    which is a list of subscribed IMAP folders, if it exists.
	  </para>
	</listitem>

	<listitem>
	  <para>
	    Updating any
	    <application>maildrop</application>
	    mail filtering recipes,
	    <filename>$HOME/.mailfilter</filename>, if it exists, to reference
	    the unicode maildir folders; or updating any custom site mail
	    filtering engine that delivers to maildir folders, to reference
	    the correct subdirectory names.
	  </para>
	</listitem>
      </itemizedlist>
    </refsect2>
    <refsect2 id="unicodesteps">
      <title>Unicode conversion steps</title>

      <para>
	The <option>--checkutf8</option> and
	<option>--convutf8</option> options to
	<command>maildirmake</command> convert a single maildir to the new
	unicode format:
      </para>

      <blockquote>
	<informalexample>
	  <programlisting>
$ ./maildirmake --checkutf8 ~/Maildir ~/.mailfilter
Checking /home/mrsam/Maildir:
Mail filter to INBOX.R&amp;AOk-sum&amp;AOk- updated to INBOX.Résumé
Subscription to INBOX.R&amp;AOk-sum&amp;AOk- changed to INBOX.Résumé
Rename INBOX.R&amp;AOk-sum&amp;AOk- to INBOX.Résumé
Verified /home/mrsam/Maildir/courierimapsubscribed
Verified /home/mrsam/.mailfilter
$ ./maildirmake --convutf8 ~/Maildir ~/.mailfilter
Checking /home/mrsam/Maildir:
Mail filter to INBOX.R&amp;AOk-sum&amp;AOk- updated to INBOX.Résumé
Subscription to INBOX.R&amp;AOk-sum&amp;AOk- changed to INBOX.Résumé
Rename INBOX.R&amp;AOk-sum&amp;AOk- to INBOX.Résumé
Updating /home/mrsam/Maildir/courierimapsubscribed
Updating /home/mrsam/.mailfilter</programlisting>
	  </informalexample>
      </blockquote>

      <para>
	<option>--checkutf8</option> goes through the motions of converting
	a single maildir to Unicode, but without making any actual changes.
	<option>--convutf8</option> does the conversion for real.
	The first required parameter is the maildir to convert. The
	second parameter is optional, and specifies the corresponding
	<command>maildrop</command> filtering recipe,
	<emphasis>but only if <application>SqWebMail</application></emphasis>
	generates the mail filtering recipes.
	<application>SqWebMail</application>'s mail filtering recipes are
	parsable, and can be automatically-converted.
	Non-<application>SqWebMail</application>-generated
	<filename>.mailfilter</filename>s cannot be converted automatically.
	The second parameter must be omitted, and the mail filtering recipe
	must be converted by hand.
      </para>

      <note>
	<para>
	  All this work is only needed if maildirs have folders with
	  non-English names. Ignore everything you've just read if all
	  folder names are English-only.
	  <option>--checkutf8</option> and
	  <option>--convutf8</option> will not do anything, and nothing
	  needs to be done.
	</para>
      </note>

      <para>
	To convert all mailboxes to Unicode all at once:
      </para>

      <itemizedlist>
	<listitem>
	  <para>
	    A shell script needs to run the <option>--checkutf8</option>
	    option for every mailbox. A list of all accounts' maildirs
	    needs to be prepared in advance, together with the corresponding
	    <filename>.mailfilter</filename>s (where appropriate).
	    courier-authlib's <command>authenumerate</command> command is usually
	    a good starting point.
	    It's ok to explicitly specify each mailbox's
	    <filename>.mailfilter</filename>, when using
	    <application>SqWebMail</application> even if a particular
	    mailbox does not use it. It will be ignored.
	    The list of all accounts' maildirs gets converted to a shell
	    script that runs <command>maildirmake</command> with the
	    <option>--checkutf8</option> option. The script should report
	    any maildir whose <option>--checkutf8</option> option reports
	    an error, and
	    <command>maildirmake</command> exits with a non-zero status.
	  </para>

	  <para>
	    It is safe to run <option>--checkutf8</option> without shutting
	    down your mail server. A non-zero exit from
	    <option>--checkutf8</option> indicates a problem (see below)
	    for a particular maildir.
	  </para>
	</listitem>

	<listitem>
	  <para>
	    Once <option>--checkutf8</option> does not find any problems
	    with any mailbox, shut down the mail server, run
	    <option>--checkutf8</option> one more time for all mailboxes,
	    then if everything goes well, upgrade
	    <application>Courier</application>,
	    <application>Courier-IMAP</application>, or
	    <application>SqWebMail</application> and
	    run
	    <option>--convutf8</option> on every mailbox before restarting
	    the server.
	  </para>
	</listitem>
      </itemizedlist>

      <note>
	<para>
	  <option>--convutf8</option> is a one-shot deal. Do not run
	  <option>--convutf8</option> a second time after it successfully
	  converted a maildir. In nearly all cases nothing will happen,
	  but there are rare edge cases where some folder names may
	  get garbled, or it fails completely.
	</para>
      </note>
    </refsect2>

    <refsect2 id="unicodeconvproblems">
      <title>Resolving unicode conversion problems</title>

      <para>
	The only likely problems that might be encountered is the fall-out
	from buggy IMAP clients that did not follow the pre-Unicode naming
	convention for non-Latin folder names. The customized IMAP
	<quote>modified-UTF7</quote> encoding convention for non-Latin
	folder names is mostly an IMAP client convention, and the
	pre-Unicode version of <application>Courier-IMAP</application> did
	not enforce it. The server took the name from the IMAP client,
	as is.
      </para>

      <para>
	Unicode conversion (<option>--checkutf8</option> or
	<option>--convutf8</option>) fails if it finds a folder name that
	does not correctly use IMAP's
	<quote>modified-UTF7</quote> encoding. This can only be resolved
	manually, by renaming the folder. This may also involve manually
	editing <filename>courierimapsubscribed</filename> and
	<filename>.mailfilter</filename> if they exist. The bad folder name
	should be removed from
	<filename>courierimapsubscribed</filename>. For
	<filename>.mailfilter</filename> it is sufficient to remove only
	the comments that precede the actual <command>maildrop</command> rule,
	and <option>--convutf8</option> will remove the entire rule, by itself.
	<option>--convutf8</option> actually reads only the machine-parsable
	comments in <command>SqWebMail</command>-generated
	<filename>.mailfilter</filename> (plus a few other things in the
	file), and replaces the
	<filename>.mailfilter</filename> with the Unicode version based
	solely on the parsed data.
      </para>
    </refsect2>

    <refsect2 id="unicodeafter">
      <title>After the Unicode conversion</title>

      <para>
	The current, Unicode version of <application>Courier-IMAP</application>
	supports both Unicode and non-Unicode
	IMAP clients; however unlike the pre-Unicode version,
	<application>Courier-IMAP</application> rejects requests from
	non-Unicode IMAP clients to use or create folders that are not
	properly encoded.
      </para>

      <para>
	Encountering a bad folder during conversion strongly suggests the
	presence of an IMAP client that does not correctly encode non-English
	folder names. Such an IMAP client will likely have problems after
	the conversion.
      </para>
    </refsect2>
  </refsect1>
  <refsect1 id="seealso">
    <title>SEE ALSO</title>

    <para>
<ulink url="maildir.html"><citerefentry><refentrytitle>maildir</refentrytitle><manvolnum>5</manvolnum></citerefentry></ulink>,
<ulink url="maildiracl.html"><citerefentry><refentrytitle>maildiracl</refentrytitle><manvolnum>1</manvolnum></citerefentry></ulink>,
<ulink url="maildirkw.html"><citerefentry><refentrytitle>maildirkw</refentrytitle><manvolnum>1</manvolnum></citerefentry></ulink>,
<ulink url="maildirwatch.html"><citerefentry><refentrytitle>maildirwatch</refentrytitle><manvolnum>1</manvolnum></citerefentry></ulink>,
<ulink url="maildrop.html"><citerefentry><refentrytitle>maildrop</refentrytitle><manvolnum>1</manvolnum></citerefentry></ulink>,
<ulink url="maildirquota.html"><citerefentry><refentrytitle>maildirquota</refentrytitle><manvolnum>7</manvolnum></citerefentry></ulink>,
<ulink url="deliverquota.html"><citerefentry><refentrytitle>deliverquota</refentrytitle><manvolnum>8</manvolnum></citerefentry></ulink>,
<ulink url="maildropfilter.html"><citerefentry><refentrytitle>maildropfilter</refentrytitle><manvolnum>7</manvolnum></citerefentry></ulink>.
</para>
  </refsect1>

</refentry>