Indic Layout Requirements

1.1 About this document

This document describes the basic requirements for Indian Languages layout for display purpose. It discusses some of the major layout requirements in first letter pseudo-element, vertical arrangements of characters, letter spacing, text segmentation, line breaking rules in Indic languages.

The current document focuses on Devanagari, but there are plans to widen the scope to encompass additional Indian scripts as time goes on.

The minimal requirements presented in this document for Indian languages text layout will also be used in E-publishing and CSS standards. This documents covers major issues of e-content in Indian languages in order to create a standard format of text layout to address storage, rendering problems, vertical writing, letter spacing, line breaking etc.

It also describes a set of ABNF-based rules for valid segmentation of Indic orthographic syllables in order to get the proper display in browsers. Text segmentation[UAX29] and line breaking [UAX14] algorithms are considered in detail. Standards for CSS and digital publications will benefit from this document.

1.2 Indian language complexities

India has large linguistic diversity with 22 constitutionally recognized languages and 12 scripts.This document is currently focused largely on the Devanagari script. The expectation is that over time its scope will widen to cover additional major scripts from the list below.

The mapping between languages and scripts is complex. Multiple languages may have common scripts, while a language can be written in multiple scripts. Each language and script is unique in nature and cannot be easily replicated, even if they share common characteristics. The orthographic changes may also occur in some languages and adoption of new orthography is a gradual process, thus posing additional challenges.

The scripts of South Asia share so many common features that a side-by-side comparison of a few will often reveal structural similarities, even in modern letter forms. They are all abugidas in which most symbols stand for a consonant with an inherent vowel.The North Indian branch of scripts was, like Brahmi itself, mainly used to write Indo-European languages such as Pali and Sanskrit, and eventually the Hindi, Bengali, and Gujarati language, though it was also the source for scripts for non-Indo-European languages such as Tibetan, Mongolian, and Lepcha, as well as many South-East Asian scripts. The South Indian scripts are also derived from Brahmi and, therefore, share many similarities in structural characteristics. For more details visit [South-Asian-Scripts].

Figure 1 shows the evolution of Indian scripts over a period of time from Brahmi script.

1.3 Basic components of Indian languages

1.3.2 Unicode Normalization

Unicode normalization [UAX15] is a form of text normalization that transforms equivalent sequences of characters into the same representation. Unicode normalization is important in Unicode text processing applications, because it affects the semantics of comparing, searching, and sorting Unicode sequences

When a unique representation is required , a normalized form of Unicode text can be used to eliminate unwanted distinctions. The key part of normalization is to provide a unique canonical order for visually non distinct sequences of combining characters.

1.3.2.1 Canonical & Compatible Equivalence

Unicode contains numerous characters to maintain compatibility with existing standards, some of which are functionally equivalent to other characters or sequences of characters. Because of this, Unicode defines some code point sequences as equivalent. Unicode provides two notions of equivalence: canonical and compatible.

Canonical equivalence is a form of equivalence that preserves visually and functionally equivalent characters.

Figure 2 shows the canonical equivalence:

1.3.3 Unicode Code charts – Devanagari & Devanagari Extended

The following Unicode Character Code chart is per the Unicode Standard:

Note

The latest version of Unicode online code charts are available at [Code-Charts] . The charts cover all character content reference of the 12 scripts of Indian languages. These files contains an excerpt from the character code tables and list of character names for the latest version of Unicode Standard.

2.1 Need for ABNF valid segmentation

ABNF Valid Segmentation based Indic orthographic syllable definition is provided here for correct and standardized representation of Indian languages layout. This will address various issues mentioned in the following sections.

This definition will be useful in order to get the uniform display of Indic layout in the browsers, applications, Digital publishing etc.

2.2 ABNF based definition of Indic orthographic syllable

V[m] |{CH}C[v][m]|CH

The linguistic definition of Indic orthographic syllable has been mapped to ABNF (Augmented Backus–Naur Form) for the purpose of text segmentation, line breaking , drop letter, letter spacing in horizontal text and vertical text representation. The definition has been elaborated , using examples from various Indic scripts in the table below.

The definition is a combination of 3 rules :

Rule 1 : V[m]

Rule 2 : {CH}C[v][m]

Rule 3 : CH (This rule is applicable only at the end of the word)

2.3 Various example use cases of ABNF based Indic orthographic syllable definition for Indian languages

Rule 1 : V[m]

3.1 Word Boundaries

Solution for word boundaries:
User-percieved characters boundaries should be based on tailored Grapheme Cluster Boundaries to conform Indic orthographic syllable definition

In case of Devanagari phrase separator । , U+0964 DEVANAGARI DANDA, (called purna viram in Hindi) and ॥ , U+0965 DEVANAGARI DOUBLE DANDA, (deergh viram in Hindi) used to mark end of the verse as in Sanskrit text, shlokas etc.),In some of the browsers ending word is selected with purna viram on double-click while in some browsers purna viram is selected as a separate.So the properties of purna viram and deergh viram should be same as the properties of FullStop or other punctuation marks so that new line should not begin with purna viram and deergh viram.

For others characters, the text segmentation should be done as Indic orthographic syllable.

3.2 Typographic units

Indic script behavior in initial letter styling is based on syllables, rather than individual letter forms.

The above Figure shows an example of a drop intial in Hindi. In the first word of the paragraph, स्कूल ('skūl'), the sequence of characters is stored in memory is as follows:

There are two syllables in this word: SA+VIRAMA+KA+UU and LA. Note, however, that there are three Unicode grapheme clusters here: SA+VIRAMA, KA+UU and LA.

Styling is done on the basis of the whole orthographic syllable, not the first character, nor even the first grapheme.

4.1 Hyphenation

There are different cases of hyphenation, some of the cases are given below :

Case 1 : Hyphens are commonly used in Copulative compounds words in Hindi language. Hindi has both prefixes and suffixes which are joined to words with a hyphen.

नर-नारी, लाभ-हानि, माता-पिता, ऊंच-नीच

4.2 Guiding principles of Line breaking for Indian languages

In Indic writing system , it is preferred that line breaks at word boundaries ,if required following principles may be adhered :

Rule 1: New line cannot begin with following symbols/Punctuation marks. Also these should be retain with the associated text

Rule 3: The hyphenated words can be broken at the hyphen e.g.:

• नर-नारी should be treated as:
• नर- on the first line and नारी on the next line

Rule 4: Expression with mathematical symbol should be treated as single unit so that at the end of the line expression should not breaks at operator level

Rule 5: Breaking should not be allowed at numerical values such as currency values, year etc. e.g.

“100.00” or “10,000”, nor in “12:59”

5.1 Selecting initial letters

Initial letters in Indic scripts must be selected on the basis of orthographic syllables, rather than individual letter forms (see an example at the end of § 3. Text segmentation). A detailed definition of Indic syllables can be found in § 2. Indic orthographic syllable boundaries. In Indian languages the size of the initial letter is determined by the number of the lines between the top line of the syllable and the lowest bit in the orthographic Indic syllable cluster where subjoined consonant and other diacritics appear.

5.2 Typical drop initial usage in Indic scripts

Most of the Indic drop initial letters in magazines and newspapers use 2 to 4 line drops. Some examples are shown below.

The sunken and raised initial letter are not preferred in Indian languages. In the above examples , reference points on the drop cap must align precisely with reference points in the text. .In Indic scripts the top reference point is the hanging base line for those scripts that have one, and the mean/median line for those that don't, and the bottom alignment point is the text after-edge

Initial letter wrap property is not applicable for Indian languages. No contour-filling is required in Indian languages.

Alignment of the top line of the non-highlighted characters at the top of the thicker top line of the initial letter is common in India. In some examples the top lines of the initial letter and the following letters don't touch. This is due to variable technology/formats used by the publishers. It is preferred that both the top lines of Initial letter and neighbouring text should touch.

Here are some additional examples of initial highlighted letter and drop letter based on the Indic syllable definition.

The remainder of this section describes the detailed rules for placement and alignment of characters with initial letter styling relative to the adjacent text.

5.3 Alignment of Initial letter of Indic scripts with hanging baseline

Indian languages which use hanging baseline such as Hindi, Bengali, Gujarati, Marathi, Punjabi etc. The part from the hanging baseline and the ascent of the Initial letter may follow the following mechanism, where n = h/2:

In Indic scripts that have a hanging baseline, the top alignment point is the hanging baseline, and the bottom alignment point is the text-after-edge, and the hanging baselines of both the initial letter and first line of text should be aligned.

5.4 Scripts that don't have hanging baseline such as Kannada, Tamil, Telugu, Malayalam , Odia etc

Publishers in India commonly used the following rules for such scripts:

• The height of the initial letter is based on the Indic orthographic syllable described in § 2. Indic orthographic syllable boundaries.
• Ascent of the first non-highlighted line is equal to the median/mean line of the initial letter as shown below :

Based on above observations the general rule for South Indian languages Indian languages scripts will be :

• ¼ height of the total Drop Cap Height projected or ascended above the ascent of the first-line
• ¾ of the total Drop Cap Height occupied or descended from ascent of first line or X1 to descent of the last line or line XN.

5.5 Initial Letter box formatting in Indian languages

When initial letters are highlighted within a box, Indian publishers commonly use different heights for the boxes and sizes for the characters. It is proposed that the syllable within the box is centre-aligned with reference to the box parameters as shown in the figure below :

7.1 Variations in vertical arrangements