I18n:Updating Unicode version

I18n:Home Page

This document describes the process of updating the files in the Mozilla codebase that are generated from Unicode data files.

Case conversion

1. Download the latest version of UnicodeData.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
2. Copy this file to intl/unicharutil/tools/UnicodeData-Latest.txt in the mozilla source tree
3. Until bug 210501 is fixed you will have to edit UnicodeData-Latest.txt by hand and delete all the lines for codepoints above FFFF
4. Run perl gencasetable.pl in intl/unichar/tools. This creates a new version of intl/unicharutil/src/casetable.h

Character properties

1. Download the latest version of UnicodeData.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
2. Copy this file to intl/unicharutil/tools/UnicodeData-Latest.txt in the mozilla source tree
3. Run perl gencattable.pl in intl/unichar/tools. This creates a new version of intl/unicharutil/src/cattable.h
4. The previous step will probably issue warnings like the following:

WARNING, Unicode Database now contain characters which we have not considered.
change this program !!!
Problem- U+010900 - U+010907 range

In this case, you will need to edit @range in gencattable.pl to include the new ranges

Transliteration

1. Download the latest version of UnicodeData.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
2. Copy this file to intl/unicharutil/tools/UnicodeData-Latest.txt in the mozilla source tree
3. Run perl gentransliterate.pl in intl/unichar/tools. This creates a new version of intl/unicharutil/tables/transliterate.properties

Bidi

1. Download the latest version of BidiMirroring.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/BidiMirroring.txt
2. Copy this file to intl/unicharutils/util/BidiMirroring.txt in the mozilla source tree
3. Run perl gensymmtable.pl in intl/unicharutils/util. This creates a new version of intl/unicharutils/util/symmtable.h
4. Download the latest version of UnicodeData.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
5. Copy this file to intl/unicharutils/util/UnicodeData-Latest.txt in the mozilla source tree
6. Run perl genbidicattable.pl in intl/unicharutils/util. This creates a new version of intl/unicharutils/util/bidicattable.h
7. The previous step will probably issue warnings like the following:

WARNING, Unicode Database now contain characters which we have not considered.
change this program !!!
Problem- U+010900 - U+010907 range

In this case, you will need to edit @range in genbidicattable.pl to include the new ranges

Default ignorable characters

1. Download the latest version of UnicodeData.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/UnicodeData.txt.
2. Copy this file to intl/unicharutil/tools/UnicodeData-Latest.txt in the mozilla source tree
3. Download the latest version of DerivedCoreProperties.txt from the Unicode website. The current version can be found at http://www.unicode.org/Public/UNIDATA/DerivedCoreProperties.txt.
4. Copy this file to intl/unicharutil/tools/DerivedCoreProperties.txt in the mozilla source tree
5. Run perl genignorable.pl | perl ccmapbin.pl - gIgnorableCCMapExt ignorable in intl/unicharutil/tools. This creates a new version of gfx/thebes/src/ignorable.x-ccmap

Other

Punctuation marks: See bug 375291 for more.