[OS X TeX] Building new formats (MacTeX)

[Gerben Wierda]

(Real info below, or at least an attempt at it).

>> On 17-sep-2006, at 22:23, Rowland McDonnell wrote:

> I cannot
> figure out anything about how to use i-installer to give me what I want,
> or indeed anything remotely useful.  I asked the author for assistance.
> I got none.

[and]

> Personal experience
> which shows that there is no useful documentation, and the author will
> not provide help to a confused user?

[and]

> Now, I've tried i-installer and just ended up with junk on my hard disc
> that I'm going to have to re-format to remove.  The documentation is
> useless, and so's the author if you ask him for help.

[but]

> It turns out that the MacTeX
> installer doesn't work properly, but while i-installer's author wouldn't
> help me sort out i-installer, he did help me sort out the problems I had
> with MacTeX.

The guy is crazy! Why can't he stay in character as a decently evil
malware provider?

[snip]

> What I need to know is what fmtutil does.

Well, as I've written in a different post, there is no decent
documentation of fmtutil, updmap and friends as far as I know, certainly
not at the level you will probably require. You might try to ask Thomas
Esser (the author of these tools) if he can provide documentation other
than the outout with --help or the few snippets in TETEXDOC.pdf.

[snip]

> Hattie:rowland rowland$ find /usr/local/teTeX  -name "language.dat"
> /usr/local/teTeX/share/texmf.gwtex/tex/generic/config/language.dat
> /usr/local/teTeX/share/texmf.tetex/tex/generic/config/language.dat
> /usr/local/teTeX/share/texmf.tetex/tex/lambda/config/language.dat
> /usr/local/teTeX/share/texmf.tetex/tex/platex/config/language.dat
>
> It's multiple files, but which are used, and for what? I cannot find
> out because I can't work out which invocation of kpsewhich applies to
> what happens when one uses fmtutil to re-create formats.

I also do not know that by heart, but in cases when I need to know, I look
in the code of fmtutil and look what options have been given to kpsewhich
when looking for language.dat, fmtutil.cnf, etc. But see below for an
attempt at explanation.

Reading kpathsea.pdf does help with texmf.cnf and path seraching. I
consider kpathsea.pdf pretty decent documentation. YMMMV of course.

>> > I want them in my local texmf tree, not texmf.local where they're
>> > likely to be over-written by something automatic in the future.
>>
>> Someone else may try to add that.
>
> It's been done, and I'm using it.

Very wise. While the TeX i-Package leaves texmf.local alone (except when
leaving it alone would mean a possible non-working TeX), MacTeX (what you
prefer using) overwrites texmf.local

[snip]

> Read lthyphen.dtx and the rest of the LaTeX source code if you want more
> information.

LOL ;-)

>> >> There are two files in most text distributions (well, unix-
>> >> distributions anyway) that control everything: texmf.cnf and
>> >> fmtutil.cnf.
>> >
>> > Except, of course, that there are multiple files called texmf.cnf
>> > and fmtutil.cnf, aren't there?  So the number is higher than two,
>> > and confusion reigns as a result.
>>
>> There is a hierarchy in the texmf.cnf files, with each level adding
>> bits, that is at least my understanding.
>
> As I understand it, at any given call to a texmf.cnf file gives you one
> file only.

The answer in in the already mentioned (three times in these threads?)
kpathsea.pdf.

texmf.cnf is the exception to the rule of kpathsea searching. All
texmf.cnf files in the search hierarchy (see below) are being read.
Settings in each of them apply if the setting has not been made in an
earlier read file. So, the programs reads (currently and with a TeX from
MacTeX/TeX i-Package):
    /usr/local/teTeX/texmf.cnf
(which also sets up the basic search hierarchy in the first 50% of the
file or so, the 4 texmf directories in the example in the next 4 lines)
and then tries to read in order:
    /usr/local/teTeX/share/texmf.local/web2c/texmf.cnf
    /usr/local/teTeX/share/texmf.gwtex/web2c/texmf.cnf
    /usr/local/teTeX/share/texmf.tetex/web2c/texmf.cnf
    /usr/local/teTeX/share/texmf/web2c/texmf.cnf
(any of these that do not exist are just silently skipped). If something
exists in an earlier one, it is ignored when encountered in a later one
(as documented in kpathsea.pdf where also SELFAUTOPARENT is explained).

Now here is also a good place to explain a bit about fmtutil and friends
(as far as my knowledge goes). When handling config files like
fmtutil.cfg, updmap.cfg, language.dat, the fmtutil/texconfig/updmap tools
use kpsewhich to find the files and/or locations. For instance for
updmap.cfg:
    kpsewhich --format='web2c files' fmtutil.cfg
is used to find the location of the current fmtutil.cfg. If this file
needs to be changed it needs to be available from the location where TeX
is configured to keep these TeX-administrator changed files (VARTEXMF).
This location is configured (in texmf.cnf) to be (TEXMFLOCAL)
    /usr/local/teTeX/share/texmf.local
If that file does not exist yet, fmtutil first copies the current 'active
updmap.cfg', e.g.
    /usr/local/teTeX/share/texmf.tetex/web2c/updmap.cfg
to
    /usr/local/teTeX/share/texmf.local/web2c/updmap.cfg
which then becomes the new 'active' updmap.cfg (the one found first by
kpathsea, thus hiding all the others which are found later, like the one
in texmf.tetex)

What updmap-sys --edit does therefore is:
1. Find the location of the current active updmap.cfg
2. Find the location of the texadmin-maintained updmap.cfg (in VARTEXMF)
3. If 2 is empty, copy 1 to 2
4. Open 2 for editing in a command line editor (in fact I think you could
tweak matters such that you would open a GUI-editor like TextEdit.app (but
I never tried), but I guess that if you set your EDITOR command line
environment variable to something like "open -a TextEdit.app" it would
probably load the file in TextEdit.app)
5. After you have quit the editor, find out if you have changed the file
and if you have do 6a and otherwise 6b
6a. run the updmap script to use the changed settings to create new map
files for dvips, pdftex and dvipdfm.
6b do nothing

A simple updmap call runs just 6a.

A call like "updmap-sys --enable Map foo.map" runs 1,2 and 3, then changes
texmf.local/web2c/updmap.cfg making sure there is a line with foo.map and
then runs 6a.

A call like "updmap-sys --listavailablemaps" runs 1, scans the resulting
file for lines with active map files, checks if these can be found with
kpsewhich and report the result.

[snip]

>> > I do know that:
>> >
>> > Hattie:teTeX rowland$ kpsewhich language.dat
>> > /usr/local/teTeX/share/texmf.gwtex/tex/generic/config/language.dat
>> >
>> > and
>> >
>> > Hattie:teTeX rowland$ find . -name "language.dat"
>> > ./share/texmf.gwtex/tex/generic/config/language.dat
>> > ./share/texmf.tetex/tex/generic/config/language.dat
>> > ./share/texmf.tetex/tex/lambda/config/language.dat
>> > ./share/texmf.tetex/tex/platex/config/language.dat
>> >
>> > which doesn't help me much.  What gets used for what, and when, and
>> > why? I don't know and can't find out yet.

Which file is found depends on two things (in the following order):
1. Which program is trying to find it (e.g. pdftex, tex, dvips, platex)
2. In which tree it located (e.g. texmf.tetex, texmf.local,
/Users/texmf.rjmm)

Try the following commands:

kpsewhich -progname=platex language.dat
kpsewhich -progname=lambda language.dat
kpsewhich -progname=pdftex language.dat

These commands emulate respectively platex, lambda and pdftex trying to
find language.dat. platex will probably find
    [...]texmf.tetex/tex/platex/config/language.dat
lambda will probably find
    [...]texmf.tetex/tex/lambda/config/language.dat
pdftex (and pdflatex, tex, latex, etc) will probably find
    [...]texmf.local/tex/generic/config/language.dat
Note that your find command above was not exercized on the tree where your
actula active TeX lives, or find would also had found language.dat in
texmf.local. (I type [...] because I am lazy).

>> The arguments to the -ini call as used by fmtutil and by kpsewhich
>> are the same. Does that help?
>
> Not in the slightest.  I think you're playing some kind of intellectual
> game with me, trying to feed me hints to point me in the direction you
> think I ought to be going.  Well, you're wasting my time and yours in
> doing so.  I can't follow you.

If you can't follow, fine. But if you ask people in effect "where should I
ought to be going" (which is how most of us would interpret a call for
directions) you can expect an answer that points you in a certain
direction. People tend to try to do what they think you are asking.

As I have gathered so far, you normally start asking for 'directions'and
if you do not like or immediately understand the directions given, you ask
for a definitive map (= good documentation so you can learn to make your
own directions). If good documentation is not available (as is the case
with commands like updmap), directions ("where you ought to go") are
however the best you can expect. If it is given (by people trying to help,
mind), it does not do to reply (with a negative connotation) that you
think they are playing a game with you.

(incidentally, have a look at game playing in the start of Wittgenstein's
"Philosophical Investigations" and you might conclude there is almost
nothing *but* game playing in human interaction, but I digress).

>> When you run fmtutil, or pdfetex -ini, you run as a super user, not
>> yourself. This means that TeX will not find the configurations files
>> at all, and will still use the default files.
>
> How can running as superuser *prevent* access to part of the file
> system?  Or rather, what's going on?

If you run as superuser with the "sudo -H" command, the command changes
the home directory to the home directory of the  user you are running as
(i.e. root). Most likely there is no Library/texmf in root's home
directory. But since you are neither using texmf.local nor
~/Library/texmf, the point is moot. I think your /Users/texmf.rjmm is
absolutely the best solution for you.

> I don't see any reason why one shouldn't point TeX at one's home
> directory as a place to look for config files, and if that's done -
> well, it'll look there when you're superuser as well as `yourself',
> surely?

If you run
    sudo -H fmtutil-sys
it will look in root's home dir

If you run
    sudo fmtutil-sys
it will look in your home dir

One issue is that
    sudo -H fmtutil-sys
will give the same result, no matter which user is running it (which I
think is preferable for a command with system-wide implications), while
    sudo fmtutil-sys
depends on who is running it.

Basically, this part of TeX Live/teTeX is far from ideal in my opinion.
And especially since the changes in 2004/2005 made it incompatible with
what was normal before, making a lot of old documentation wrong for the
current situation.

G

------------------------- Info --------------------------
Mac-TeX Website: http://www.esm.psu.edu/mac-tex/
          & FAQ: http://latex.yauh.de/faq/
TeX FAQ: http://www.tex.ac.uk/faq
List Archive: http://tug.org/pipermail/macostex-archives/