[Textures] A bit of everything (1/7)

[Bruno Voisin]

Hello,

Over the years 2001 to 2008, as a regular contributor to the TeX on Mac OS X mailing list <http://mactex-wiki.tug.org/wiki/index.php/TeX_on_Mac_OS_X_mailing_list>, I often dealt with font-related issues either raised on the list or sent to me off-list. Most of the time this was done in haste and ended in half-baked solutions, which sort of worked.

A few months ago I decided to go back to this and tie up the loose ends, for those solutions I had been using myself afterwards. The immediate motivation was the purchase of up-to-date versions of the Lucida and MathTime fonts from PCTeX, to replace the versions purchased from Blue Sky Research in the mid-1990s and used extensively ever since (especially Lucida). The new versions needed to be installed in MacTeX, which required cleaning the accumulated stuff in /Library/TeX/Local and ~/Library/texmf. While I was at it, I decided to adapt the results to Textures as well, which remains my TeX front-end of choice.

Needless to say, this was a bad decision and turned out to be far more difficult and take far much time than expected. This started in early October 2010, has been carried at intervals since, and has only just been completed!

In the process a lot of issues were found and, hopefully, solved. Since this may be of interest to members of this list, here is an account of what I did. First come general principles about how to install fonts and metrics in Textures, then the particular cases I dealt with, in chronological order which is also roughly an order of increasing complexity.

A parallel series of messages will be sent to the TeX on Mac OS X list, describing similar adaptations for MacTeX. The following assumes the reader has installed MacTeX <http://www.tug.org/mactex/>, of which several components will be used.


General principles: Fonts
=========================

TeX fonts generally come in PostScript Type 1 format (either PFB binary files or PFA ascii files), often accompanied with metric files in AFM format (if not, the AFM files can be recreated from the PFB or PFA files). To be used by Textures, the fonts must be converted to a format recognized by OS X, which may raise license issues. I won't enter into these, except in one case (Lucida), and leave it to the reader to assess the matter.

PostScript
----------

The Mac version of a PostScript font is a screen font suitcase, in FFIL resource format or DFONT data format, plus printer font files in LWFN format. These are legacy formats inherited from Mac OS Classic. On OS X they are deprecated and may stop being supported at any point in the future (Lion?).

MacTeX includes a tool t1mac to convert from PFA/PFB to LWFN. Assuming a folder of PFB files, just cd into it in Terminal then type

	apply 't1mac %1 | macbinary decode -p -o `basename %1 .pfb`' *.pfb

The output of t1mac, which by default is in MacBinary format, has been piped to macdecode (part of OS X) for decoding into Mac OS files. t1mac has an option -r for producing what it calls "raw resource forks", but I couldn't figure out what raw resource forks are and how to use them, whereas the above just works.

In addition, FFIL/DFONT screen fonts are required. These are only used by OS X to locate the LWFN files and get basic metric info, no character bitmaps or outlines from the screen fonts are actually used. For that reason, the Windows-only Y&Y Font Manipulation Package (see "Creating Mac screen fonts from Type 1 outlines" at the bottom of <http://www.faqs.org/faqs/fonts-faq/part7/>) included a tool AFMtoSCR for creating fake Mac screen fonts from AFM metric files.

The output of AFMtoSCR was a MacBinary-encoded SCR file, which needed to be decoded afterwards. For a folder of SCR files, this means

	apply 'macbinary decode -o `basename %1 .scr`.bmap %1' *.scr

The extension of the result is immaterial (for such legacy file formats, OS X ignores the extension and uses the Classic "type" flag), I have used .bmap which is the extension used by FontForge.

The European Modern fonts (see the last message in this series) are provided with SCR files. I am not aware of any other example. AFMtoSCR still lives as C source <http://code.google.com/p/yytex/source/browse/trunk/src/c32/afmtoscr.c>, but I do not think it has ever been ported to the Mac or to Unix.

This only leaves FontForge <http://fontforge.sourceforge.net/>, using the binary packages now relegated at <http://sourceforge.net/projects/fontforge/files/fontforge-executables/>. FontForge can be scripted, and the GW Extras i-Package includes such a script dumpafm.pe for extracting AFM files from a DFONT suitcase or a TTC collection (see the penultimate message in this series). However, I am totally helpless at scripting and won't venture into this territory.

So let's go GUI: with FontForge installed, practically any font file can be double-clicked in the Finder and it will open in FontForge. For each PFB file you must first create a bitmap version by going to

	Element > Bitmap Strikes Available

(namely, go to the Element menu and select "Bitmap Strikes Available"). Check the Mac checkbox and enter a bitmap size; I chose 4 pt, which was the bitmap size of the screen fonts in BSR's distribution of the Lucida fonts (this is immaterial, since the bitmaps are not actually used). Then go to

	File > Generate Fonts

and select

	PS Type 1 (Resource) + NFNT (Resource)

This will create a .bmap screen font in FFIL format, and a LWFN printer font file. The FFIL file will have the name of the AFM metrics inside the PFB file, and the LWFN file will have a name satisfying some norm like 3-letter abbreviations for each of Bold, Italic, etc.

However, the result doesn't work because of the way the fonts are grouped in families by Mac OS. Consider the example of the Lucida Bright family, namely Lucida Bright, Italic, Oblique, Demi and Demi Italic. The above procedure yields

	lbr.pfb		-> LucidBri + LucidaBright.bmap 
	lbi.pfb		-> LucidBriIta + LucidaBright-Italic.bmap
	lbsl.pfb	-> LucidBriObl + LucidaBright-Oblique.bmap
	lbd.pfb		-> LucidBriDem + LucidaBright-Demi.bmap
	lbdi.pfbv	-> LucidBriDemIta + LucidaBright-DemiItalic.bmap	

If you put the resulting files in /Library/Fonts or ~/Library/Fonts and open Font Book, you'll realize that only the Oblique variant is seen, not the others. First failure.

Go back to FontForge, open the above 5 PFB files all at the same time, select the window for the Roman font lbr.pfb, go to

	File > Generate Mac Family

select

	PS Type 1 (Resource) + NFNT (Resource)

and make sure the checkboxes for Lucida-{Demi,Oblique,DemiItalic,Italic} are all checked. This will generate the same LWFN files as above plus one FFIL suitcase, with .res extension, for the whole family. Erase the .bmap files from /Library/Fonts or ~/Library/Fonts and put the .res suitcase there. Open Font Book: all the 5 fonts of the Lucida Bright family are now visible, except the Italic variant. Second failure.

There is indeed a workaround: if the fonts are re-encoded to Macintosh Standard Roman before creating the bitmaps, by going to

	Encoding > Reencode > Macintosh Latin

and if individual .bmap FFIL screen fonts are created as above, then all the fonts from the Lucida Bright family are seen by OS X. But this cannot apply to math fonts like lbmr.pfb (Lucida NewMath Roman), which use custom encodings and will no longer work with TeX if they are re-encoded to Macintosh Standard Roman. Third failure.

So, except for European Modern for which pre-made SCR files are available, FFIL/LWFN is no-go.

OpenType
--------

OpenType OTF format is the next candidate, called "OpenType (CFF)" by FontForge. As far as I understood this is mostly a container preserving the PostScript nature of the PFB font, namely the fact that the glyphs are defined by cubic splines.

Unfortunately in all the tests I did, trying all possible prior re-encodings, no glyph showed up in Textures when drawing char tables, or at best the 24 alphabet letters and a couple of other characters showed up but no accents, no diacritics, etc.

The reason for this remains a mystery to me: is there some step I missed or some setting I did wrong in FontForge, or is Textures 2.2 unable to use OpentType fonts?

TrueType
--------

This only leaves TrueType, which means conversion from cubic to quadratic splines and also conversion of hints from one type to another. Open the PFB files in FontForge one by one, go to

	File > Generate Fonts

then choose

	TrueType

for text fonts and

	TrueType (Symbol)

for math fonts. This produces TTF files, which is the format of many of the fonts shipped with OS X anyway. When these files are put in /Library/Fonts or ~/Library/Fonts, they are all seen and properly grouped in families in Font Book, hence accessible to Textures.

The fonts can also be grouped in families directly in FontForge, namely saved as TTC collections instead of individual TTF files. TTC is the format of several of the fonts shipped with OS X as well. Proceed as for PostScript font families above, namely open all the PFB files corresponding to one family at the same time in FontForge and select the window for the Roman font. Then go to

	File > Generate TTC

and make sure the checkboxes for all the other fonts in the family are checked. However, given the fiasco with PostScript font families above, and since things work very well with individual TTF files, I didn't experiment with collections.

Does it matter?
---------------

Based on the above, all the adaptations in this series were done converting from PFB to TTF, and thinking that the degradation associated with the conversion was purely theoretical and invisible in practice. Then, following suggestions by Paolo Matteuci, I decided at the last minute to add one more case study and look at European Modern, for which both FFIL/LWFN and TTF can be used.

Comparing the two, it turns out the difference, though small, is not fully negligible. Here are glyph charts of emr12 side-by-side at 400% magnification in Adobe Acrobat Pro 10, FFIL/LWFN on the left, TTF on the right (the charts were printed from Textures into PDF files):

-------------- next part --------------
A non-text attachment was scrubbed...
Name: TTF vs LWFN.png
Type: image/png
Size: 451222 bytes
Desc: not available
URL: <http://email.esm.psu.edu/pipermail/textures/attachments/20110322/269fb964/attachment.png>
-------------- next part --------------


You can see that the upper bars of the capital I and J are perfectly right for FFIL/LWFN and slightly curved (and blurred) for TTF. The reason for this becomes visible at higher magnification :

-------------- next part --------------
A non-text attachment was scrubbed...
Name: LWFN vs TTF Reader.png
Type: image/png
Size: 393055 bytes
Desc: not available
URL: <http://email.esm.psu.edu/pipermail/textures/attachments/20110322/269fb964/attachment-0001.png>
-------------- next part --------------


The curvature is caused by anti-aliasing, which yields small "bulges" or shades at the ends of the horizontal bars of the I and J. Since anti-aliasing differs from one PDF renderer to the other, it is not surprising that the same effect is not visible in Preview:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: LWFN vs TTF Preview.png
Type: image/png
Size: 419481 bytes
Desc: not available
URL: <http://email.esm.psu.edu/pipermail/textures/attachments/20110322/269fb964/attachment-0002.png>
-------------- next part --------------


Another reason for preferring PostScript over TrueType is licensing: it may be argued that conversion from PFB to FFIL/LWFN is not really a conversion, it is rather an extraction of the PostScript data from their original PFB packaging followed by re-packaging in LWFN form. The same reasoning was invoked to dismiss the license issues possibly associated with the creation of the GTA Mac Fonts by the GW Extras i-Package: the fonts were obtained by extracting the TrueType data from their original DFONT packaging and re-packaging them as individual TTF files (see the penultimate message in this series). 

In the same way, the degradation in quality is sometimes invoked as the reason for prohibiting automatic conversions (see 
<http://tug.org/pipermail/lucida/2010-May/000338.html> which seems to imply this for Lucida), and for conversion from PFB to LWFN there is no such degradation.

So to summarize, the following was done using the TTF format for fonts for lack of a better solution, in the hope that suggestions on this list would follow proposing better solutions. I'll come back to the license issues for Lucida in message 4 of this series.

Installation
------------

Recently Textures has introduced the ability to install fonts in Textures-specific font folders, namely /Library/Application Support/Textures/Fonts for a system-wide setup and ~/Library/Application Support/Textures/Fonts for a user-specific setup, in addition to the usual OS X font folders /Library/Fonts and ~/Library/Fonts, respectively.

This is quite convenient and makes sense when you need to install a font specifically for Textures, to avoid a clash with another version of the same font at the OS level (for example with the Lucida fonts installed by Microsoft Office inside /Library/Fonts/Microsoft) or to avoid cluttering the font menus of other GUI applications.

But beware: putting fonts inside the Textures Fonts folders slows down Textures at startup very very very significantly. On my setup, which is normally quite fast (a MacBook Pro 17" 2010 with Core i5 at 2.53 GHz, 8 GB RAM and 512 GB SSD drive), this means Textures starts in 30 seconds or even more, instead of a couple of seconds at most when the Fonts folder is empty.

Another problem is "caching": I could not find any repeatable pattern to it, but it seems that the fonts from the Textures Fonts folders are activated at the OS level, not only within Textures, and remain activated from one Textures session to the next (ie closing then reopening Textures) until you end up the current user session in the Apple menu or with Cmd-Shift-Q. This seems confirmed by the fact that if you open Font Book first, then Textures, you'll see a spinning wheel appear several times in the status bar at the bottom of the Font Book window while Textures is starting, indicating that Textures does something to fonts which Font Book notices.

So even if you close Textures and remove a font from the Textures Fonts folder before reopening Textures, then this font may still, in some circumstances I could not determine with certainty, be seen and used by Textures.

At one point I think, while I was switching to the Typeset window in Textures, a dialog popped up telling that Textures needed to activate a font outside the OS repository for fonts. I pressed OK, and checked a box telling "Don't ask again". Now I don't have a clue what triggered this dialog, and how to make it appear again. And Textures had started before that anyway, to still use fonts even after they were displaced outside the Textures Fonts folder. I tried clearing the font caches using

	sudo atsutil databases -remove

but that didn't help.