Background
The UBA's rendering of a piece of text depends not only on the explicitly declared direction in which it appears (e.g. the dir attribute value on the parent element) and the characters it contains, but also on the implicit directional properties of the characters preceding and following it. For example, in an RTL context, "john: " is displayed as "john: " when followed by "susan" (i.e. "john: susan"), but as " :john" when followed by "SUSAN" (i.e. "NASUS :john") - note the change in colon positioning.
The bidi formatting characters LRO, RLO, LRE, RLE, and PDF have particularly strong influence on what surrounds them. For example, RLO makes all text up to the next PDF behave as RTL characters, making "hello" display as "olleh".
The Problem
Most documents contain a large number of self-contained entities whose content must not influence the directional rendering of what precedes or follows them. Furthermore, the document author naively expects such an entity to be displayed visually between what precedes it and what follows it, laid out in the current direction: preceding - entity - following.
Examples of such entities are legion: the title of an article, the name of an author, a description, etc.
As long as the entire document and all the entities it contains are of uniform direction, there is no problem. Arbitrary-direction entities also don't cause a problem when they are displayed as a separate block element (which is treated as a separate "paragraph" in UBA terms). However, when an inline entity is allowed to contain text of arbitrary direction, bad things start happening, and existing HTML mark-up is powerless to stop it.
PURPLE PIZZA - 3 reviews
The entity here is the RTL name of a restaurant, being displayed in an LTR context. The intent is to have it appear as
AZZIP ELPRUP - 3 reviews
However, it is actually displayed as
3 - AZZIP ELPRUP reviews
and is effectively unreadable. This happens because according to the UBA, a number "sticks" to the strong-directional run preceding it.
<span dir="rtl">PURPLE PIZZA</span> - 3 reviews
This is a common first attempt at fixing Example 1. In fact, wrapping opposite-direction text in mark-up indicating its direction is generally a good idea, and is in many cases essential. Here, however, it makes no difference at all - the result is exactly the same as in Example 1.
That the "fix" does not work is, in fact, to be expected: the <span dir="rtl"> only explicitly states the direction of the text inside it, and does not say anything at all about what surrounds it.
In fact, the currently recommended way to fix our purple pizza is not to use mark-up at all, but to insert an LRM character (U+200E, ‎) after the PURPLE PIZZA. This prevents the RTL text from "sticking" to the number that happens to follow it. If the context had been RTL and the entity LTR, the same magic would be worked by the RLM character (U+200F, ‏). The same technique is supposed to be applied to Example 3 and Example 4 below. Unfortunately, using LRM/RLM marks like this is less than ideal, for reasons we will discuss below.
USE css (<span dir="ltr">position:relative</span>).
The entity here is a code snippet ("position:relative"), marked with a span and its LTR direction, to be displayed in an RTL context. Despite the RTL context, it is preceded by the LTR word "css" because technical terms and brand names often appear in their original Latin script in RTL text. The intent is to have it appear as
.(position:relative) css ESU
However, it is actually displayed as
.(css (position:relative ESU
This happens because the LTR word "css" before the entity "sticks" to the LTR entity according to normal UBA rules.
documents > <span dir="rtl">MY FIRST NOVEL</span> > <span dir="rtl">CHAPTER 1</span>
The entities here are folder names displayed in "breadcrumbs" in an LTR context, where two of the folder names happen to be RTL. The intent is to have it displayed as
documents > LEVON TSRIF YM > 1 RETPAHC
However, it is actually displayed as
documents > 1 RETPAHC < LEVON TSRIF YM
i.e. with the RTL folder names visually in the wrong order (and the arrow between them reversed). This happens because according to the UBA, the two RTL entities "stick" together, whether or not they are wrapped in <span>s as shown here.
joe hackerRLO: overdrawn
The entity here is the name of a user, as chosen by a malicious user to include the invisible RLO character (U+202E), followed by a status string. Obviously, the user's name is "HTML-escaped" when displayed, but this does not do anything to the RLO character. The outcome is that this is displayed as
joe hacker: nwardrevo
where the entity influenced the display of what follows it, reversing its characters. This has security implications and has surfaced on blogs. On the other hand, it does not even have to be due to malicious use, only to the inadvertently bad trimming of an overly-long string.
Currently, there is no reliable way to deal with Example 1 to Example 4 using mark-up, except by redundantly marking an entity's surroundings with the base direction, which is counterintuitive and painful to implement. The usual way to deal with Example 1 to Example 4 is to surround an entity in either LRM or RLM characters - LRM in an LTR context, and RLM in an RTL context. This prevents the entity from "sticking" to what precedes or follows it.
However, using the LRM/RLM technique has several disadvantages, particularly in a web application:
-
The LRM or RLM is being used to address a layout issue that reflects the structure of the document, i.e. to indicate the boundary of an entity. There should be a way to express it in mark-up, not magic Unicode characters. In fact, the entity is typically already surrounded by an element that either gives it style or indicates its direction; why can't the element itself be used to indicate an entity?
-
In a web application, having to add logic to choose between an LRM and an RLM is a pain, especially when the existing code layer does not happen to have easy access to the context's direction.
-
Not all search engines (e.g. the browser's own CTRL-F) are smart enough to ignore invisible Unicode characters such as LRM and RLM. This makes a document using such characters less searchable: the user searches for "A B", but does not find "A B" because there is an invisible character between them. Or, conversely, the user copy-pastes text - accidentally including the LRM or RLM character - from the page into some search box, and does not get hits in any other documents because they do not contain the LRM/RLM. In a manually-authored HTML document using a few judiciously placed LRMs/RLMs, such problems do not amount to much. In a web application, however, the simplest way to use this technique is to do it wholesale, around every inserted entity. This results in very real searchability problems. Avoiding them requires implementing quite complicated logic to decide whether the LRM or RLM is really necessary.
Furthermore, LRMs and RLMs do not help in Example 5. Nor is there any mark-up to solve it. The only current way to deal with it is for the application to either remove any LRE, RLE, LRO, RLO, or PDF characters in it, or to remove any extra PDFs and then add any missing ones at the end. This is a rarely-implemented pain in the neck.
Proposed solution
Add an element attribute to HTML used to make an inline element directionally isolated from its surroundings. A tentative name for the new attribute might be bdi, for "bidirectional isolate", as in <span dir="rtl" bdi="yes">. The attribute would take three values:
-
no, specifying no special isolation. This is the default, except in special cases indicated in the sections below.
-
yes, specifying isolation.
-
bdi, a synonym for yes. Allows specifying the attribute without a value for conciseness, e.g. <span dir="rtl" bdi>.
Applications generating HTML would use bdi routinely on elements that wrap an inserted data string (usually in conjunction with indicating its direction using the dir attribute).
The exact definition of the effects of bdi="yes" on an element:
The element, even when empty, is to be displayed as if it were surrounded with strong-directional characters of the last explicit embedding level within which it appears. In most cases, the last explicit embedding level is simply the parent element's computed direction. (The exception is when the element is between LRE/RLE and PDF characters, which is discouraged by the W3C.) For example, take:
<span dir="rtl" bdi="yes">PURPLE PIZZA</span> - 3 reviews
In a dir="ltr" element, it should be displayed the same as
‎<span dir="rtl">PURPLE PIZZA</span>‎ - 3 reviews
i.e. as
AZZIP ELPRUP - 3 reviews
-
The "imaginary" LRM/RLM characters must not actually appear in the output, e.g. for the purposes of a "copy to clipboard" operation.
-
The effects of LRE, RLE, LRO, RLO, and PDF characters appearing in the element will never extend beyond the element: unbalanced PDFs will be ignored, and missing PDFs will be assumed at the close of the element.
The use of "imaginary" characters by higher level protocols such as HTML is explicitly allowed by the UBA's section 4.3, HL5 ("Provide artificial context").
The problem
Many web applications with an RTL-language interface or an RTL-language data source need to display and accept as input both LTR and RTL data. Furthermore, the application often does not know and can not control the direction of the data.
For example, an online book store that carries books in many languages needs to display the original book titles regardless of the language of the user interface. Thus, a Hebrew or Arabic book title may appear in an English interface, and vice-versa. The direction of the title may be available as a separate attribute, but more likely it isn't, and needs to be guessed. The safest guess is on the basis of the characters making up the title.
If this site also allows user comments or reviews, it is unreasonable to limit these to one language. For example, for an English book listed in an Arabic or Hebrew interface, it is perfectly reasonable to get comments both in English and in the book's language. The application does not know what the user will type until the user types it.
Unless opposite-direction data is explicitly declared as such, it is often displayed garbled as shown above. Perhaps even worse, the user experience of typing opposite-direction data is quite awkward due to the cursor and punctuation jumping around during data entry and difficulty in selecting text.
Currently, avoiding such problems requires that the application implement logic to estimate the data's direction - and use it in the many places where it is needed. Such logic is not easy to implement, since it requires using long tables of strong-RTL and/or strong-LTR characters, and becomes non-obvious when a string contains both. For an input element, where the direction must be automatically set as the user types the text, there is no choice but to implement the estimation logic in page scripts, thus requiring even more advanced programming skills. As a result, few applications wind up doing direction estimation, and a poor user experience is quite common for web pages mixing LTR data in an RTL interface or vice-versa.
Not the problem
The issue at hand is with text data that is basically compatible with the UBA. That is, given the correct base direction, applying the UBA will display the text intelligibly. The only problem is that we don't know the correct base direction.
This is distinct from a different, harder issue: text mixing LTR and RTL without using the formatting characters necessary to display it intelligibly using standard UBA rules. Whichever base direction is applied, the text will not be displayed as intended. Examples of such data are not as rare as one might think:
- Path or URL that includes consecutive RTL folder or file names (one would expect the path components to proceed in a uniform direction)
- "Tweets" that include both an RTL phrase and LTR parts like @name and a URL
- An RTL sentence that attempts to give a phone number with spaces in it
- Sentence containing an opposite-direction quotation that starts with a number or ends with punctuation
- Multi-paragraph text containing both LTR and RTL paragraphs, e.g. an RTL restaurant review followed by restaurant address in Latin script.
Such text does not include the Unicode formatting characters that could fix its display either because it must conform to a syntax that would misinterpret such characters, or simply because it was created by a human user that does not know such characters exist, much less how to enter or use them. Given the text's syntax, or at least a set of patterns for the problematic parts, the text could, in theory, be parsed into its constituent parts, and formatting characters added to make the text display correctly.
Although this is a painful real-world problem, it is unrelated to HTML per se and currently lacks a mature solution. We are not proposing one here.
Estimation algorithms (skip if not interested)
A data string's direction is obvious when it contains either LTR or RTL characters, but not both. The following heuristic algorithms have been used when the data does contain both LTR and RTL characters:
- First character with strong direction. This is the algorithm specifically mandated by the UBA for choosing a paragraph's base direction (unless overridden by a higher-level protocol, which is what currently always happens in HTML). This has the advantage of being easy to understand (and even surmise) for the user, and text is usually more readable when starting with a word in its overall direction. Nevertheless, it is not uncommon for an RTL phrase to start with an LTR word like a brand name or a technical term, in which case this algorithm fails.
- Does the string contain any RTL characters? This fails for LTR text that includes some RTL, which is quite uncommon, but not unheard of.
- Word count: does the percentage of RTL words exceed some threshold value? Works very well, but also proves unintuitive to the user in some circumstances.
Different approaches have been preferred in different contexts: first-strong for search boxes, any-RTL for advertisements, and word-count for longer texts like e-mails. Nevertheless, it is worth pointing out that the choice of the precise algorithm is an optimization. For most real-world data strings, all these estimation algorithms will give the same correct result.
In addition to the basic algorithm choice, there are also several side issues:
- Sometimes, there may be good reason to want to bias the estimation to a particular direction unless the actual value clearly indicates otherwise. For example, the value may be one title from a feed whose contents are, generally speaking, in a particular known language.
- When the value contains no strong-directional characters, it usually seems best to display it in the same direction as its surroundings, i.e. inherit the direction.
- How should those parts of the string bracketed in LRE / RLE / LRO / RLO and PDF characters be treated? It is possible to simply ignore such bracketing characters, and this is actually specified by the UBA for its first-strong algorithm. Another possibility is to ignore them together with the substrings they bracket. The rationale for the latter approach is that the direction we estimate for the whole string will not be applied to the bracketed substrings anyway. In fact, if part of a string is explicitly declared LTR, it is usually because the overall string is RTL, and vice-versa. On the other hand, if the string contains no strong-directional characters outside the declared substrings, and all the declared substrings give the same direction, then it might be best to estimate the string overall to be of the same direction as the declared substrings.
Proposed solution
Make simple direction estimation functionality available in the browser by allowing the dir attribute to take on new values indicating that the user agent is responsible for estimating the direction of the element's contents.
One such dir attribute value would specify using the word-count algorithm, defined and discussed in Appendix A. Another would specify the first-strong algorithm, as defined by the UBA.
Specifying one of these direction-estimation values for the dir attribute would direct the user agent to examine the element's text content and estimate whether it is LTR or RTL using well-defined heuristics based on the inherent direction of the characters (as defined by the Unicode standard). The result it will return for text mixing LTR and RTL characters, although well-defined, may or may not be correct as judged by a human user.
Although specifying direction estimation would be allowed on any element, it is primarily intended for elements wrapping a "single-origin" piece of text, e.g. a text input. The more complex the element's structure, the higher the chances that it mixes LTR and RTL content, and the lower the chances that an estimation algorithm will succeed in displaying the contents intelligibly. It is meaningless to use an estimation algorithm on content mixed to the extent that it is unintelligible in both LTR and RTL (when displayed by standard UBA rules).
The new dir values should also be added to the CSS direction property's repertoire for completeness. However, since W3C guidelines recommend that direction be declared using the dir attribute, not CSS, this is first and foremost an HTML issue.
Further details:
- The content to be examined by a direction estimation algorithm is all descendant text nodes, visited in "in-fix" order, except for those under a descendant element with a unicode-bidi style other than "normal", e.g. a <bdo> element or an element with a dir attribute (whatever the value).
- No attempt will be made to exclude "hidden" content, whether using display:none style or any other invisibility technique.
- When none of the examined content contains strong-directional characters, the parent element's computed direction will be used.
- Elements with dir values specifying estimation will be considered to have bdi="yes" by default.
- For backward compatibility, dir values specifying estimation will not be the default for any currently defined elements except as defined below.
- Ideally, the current computed direction of an auto-directional element should be exposed to scripts as an element property, e.g. computedDirection.
Open issues
- What dir attribute values should be used to specify the word-count and first-string estimation algorithms? One possibility would be simply "word-count" and "first-strong". Or should they both start with the word "auto", i.e "auto-word-count" and "auto-first-strong".
- Is it really truly essential to support both the word-count and the first-strong algorithm? Using just one algorithm would reduce confusion; there would be just one new dir attribute value, the easy-to-understand "auto".
- Should direction estimation be the default for <input type="text"> and <textarea> elements? Although it would be very useful in most cases, it is not backward compatible. The best way to come to a decision might be to discuss specific examples where estimation might prove harmful, and to judge their importance.
Background
In many applications, it is necessary to allow the user to enter text of either direction into a given <input type="text"> or <textarea> element, regardless of the page's direction.
Although algorithms for estimating the direction of a string exist (and hopefully will be exposed by the browser as described in Section 2.2 ), they remain heuristic for mixed-script strings.
As a result, all major browsers provide some way for the user to explicitly set the direction of an <input type="text"> or <textarea> element, e.g. via keyboard shortcuts, so the text being entered by the user is displayed correctly.
The problem
Once the text entered by the user has been submitted to the server, the direction in which it was displayed in the page is lost, unless explicitly added to the form as an invisible input by page scripts. However, scripts are not available in all environments, e.g. e-mail forms. As a result, in such an environment, the application is forced to guess at the direction of a string submitted by the user, will sometimes get it wrong, and as a result display it incorrectly in subsequent pages.
Proposed solution
Support a new attribute in <input> and <textarea> that would specify that the value of the element's computed direction at submission time will be included in the submission as an additional field. The additional field's name will be the value of element's name attribute suffixed with "_dir". (Reminder: the computed direction is the bottom-line "ltr" or "rtl" being used to display the element; it never takes on any other value.)
The new attribute could be called submit_dir, and would take three values: "yes", "no", and "submit_dir". The last would be a synonym for "yes", and would allow using the attribute without an explicit value, for short. The default would be "no". For example, let's assume that a dir attribute value to indicate direction estimation is "auto", and an RTL page contains the following form:
<form action="foo" method="get">
<input type="text" name="mytest" submit_dir="yes" dir="auto" />
</form>
Then, if the user typed in the LTR value "hello", the submission URL would be "foo?mytest=hello&mytest_dir=ltr".
Background
Although most images, e.g. photos, are equally applicable to LTR and RTL pages, some images are inherently and primarily "handed" or "directional", and need to appear in a mirror image in an RTL page. Common examples include various arrow and "connector" images. A less obvious example might be star rating images: the "full" half of a half-star needs to be on the left in LTR and on the right in RTL.
The problem
Currently, the author of a page to be localized into both LTR and RTL languages is forced to create two separate versions of each "handed" image, stored in two separate files, and use one or the other depending on the page language by changing the src attribute of the <img>. This process is monotonous and error-prone.
Proposed solution
Being able to tell the browser to do such flips automatically would make it that much easier for web applications to support both LTR and RTL interfaces. Only one image would be provided by the page, and the <img> element's attributes would not even have to differ between LTR and RTL pages. The single image file provided by the page would come with instructions in the <img> element to be flipped by the browser when its parent element has RTL computed direction. (The <img> element's own dir does not count, since it is used primarily to indicate the direction of its tooltip text as specified by alt and title attributes, which need not match the surroundings.)
One possibility for such a specification would be with a new HTML attribute: hflip="no|yes|ltr|rtl".
The default "no" would mean that the image should be displayed as is, and "yes" would mean that it must be displayed horizontally mirrored. To indicate that the flip should only occur when the element's parent has RTL computed direction, the value should be "rtl", and of course "ltr" would specify the inverse.
A similar capability would be very important in CSS. For example, its "background" property is very popular, especially with "sprite" images. However, this is beyond the scope of this HTML proposal.
Background
The UBA's rendering of a piece of text depends not only on the explicitly declared direction in which it appears (e.g. the dir attribute value on the parent element) and the characters it contains, but also on the implicit directional properties of the characters preceding and following it. For example, in an RTL context, "john: " is displayed as "john: " when followed by "susan" (i.e. "john: susan"), but as " :john" when followed by "SUSAN" (i.e. "NASUS :john") - note the change in colon positioning.
The bidi formatting characters LRO, RLO, LRE, RLE, and PDF have particularly strong influence on what surrounds them. For example, RLO makes all text up to the next PDF behave as RTL characters, making "hello" display as "olleh".
In the UBA, whitespace provides almost no separation against either kind of bidi influence.
On the other hand, the UBA's sections 3.3.1 and 3.3.2 require that the bidi state be completely reset at a "paragraph break". This means that strong-directional text (e.g. letters) and explicit bidi formatting characters (e.g. RLE and RLO) in one paragraph have no effect on the formatting of the text in the next paragraph and vice-versa. This is a very high level of bidi separation.
In plain text, "newline" characters like the line feed (U+000A) and carriage return (U+000D) are commonly used both to end paragraphs and simply to wrap logical lines. The former usage needs a UBA paragraph break, while the latter usage wants no more bidi separation than other kinds of whitespace. The UBA resolves this ambiguity in favor of the paragraph break because of its importance. All common UBA implementations for plain text treat newline characters as a UBA paragraph break, in accordance with the UBA specification.
The UBA leaves the definition of a "paragraph" in higher-level protocols like HTML up to the protocol.
It is well-accepted that HTML block elements like <div> and <p> form UBA paragraphs, and this is implemented by all major browsers. Thus, whatever happens inside a block element has no effect on the bidirectional rendering of the text before it or after it.
The problem
The HTML 4 standard explicitly specifies that <br> is to be treated for bidi purposes as whitespace, and not as a UBA paragraph break. The arguments for this decision seem to be that:
Firefox and Opera follow this specification and treat <br> as whitespace for UBA purposes.
In actual usage, however, <br> is a very popular element and is used to form paragraphs at least as often as <p>, just like newlines in plain text. In fact, unlike newlines in plain text, it is almost always used for that purpose, as opposed to just wrapping a line to fit in a limited amount of space, simply because HTML normally takes care of line wrapping by itself.
As a result, Firefox's implementation of <br> as UBA whitespace, despite being in accordance with the current HTML specification, is regularly reported as a bug. It results in innocent-looking HTML like
1. his name is JOHN.<br>
2. SUSAN is a friend of his.
being rendered as
1. his name is .NHOJ
NASUS .2 is a friend of his.
Because the "JOHN.<br>2. SUSAN" forms a single RTL run despite the <br>, the "2" goes to the right of SUSAN. (Please note that wrapping the "JOHN" and "SUSAN" in separate dir="rtl" spans, i.e. "<span dir="rtl">JOHN</span>.<br>2. <span dir="rtl">SUSAN</span>", does not make any difference.)
Although this LTR example is somewhat contrived, the RTL equivalent is quite realistic because it is common for LTR brand names, acronyms, etc. to be used in RTL text:
1. IT IS IMPORTANT TO LEARN html.<br>
2. css IS IMPORTANT TOO.
which is rendered in Firefox and Opera as
html. NRAEL OT TNATROPMI SI TI .1
.OOT TNATROPMI SI 2. css
As a result, IE and WebKit treat <br> as a UBA paragraph break. Although this is not in conformance with the HTML 4 spec, the bidi separation it provides does seem to follow most users' expectations.
If IE and WebKit were to change their <br> behavior to conform to the current standard, many existing RTL HTML documents would be broken, especially given that they tend to be authored mostly with IE in mind.
While the bidi separation provided by treating <br> as a UBA paragraph separator is useful, the very strong nature of this separation (closing all open embedding levels) also creates problems. Being an inline element, <br> can be nested within an arbitrary number of other inline elements. If these inline ancestors have explicit dir attribute values of their own, should the <br> terminate their effects as UBA's definition of a paragraph separator says it should? That is what a newline in plain text does when it comes between an LRE or RLE and its matching PDF. So, should the second line in <div dir="rtl"><span dir="ltr">1. hello!<br>2. goodbye!</span></div> be displayed as RTL? That would conform to the definition of a UBA paragraph break, but would go against the spirit of HTML. This is, in fact, what WebKit currently does (although it is now being treated as a bug).
To avoid this problem, IE apparently re-opens the directional embedding levels specified on ancestor elements via mark-up (dir attribute, <bdo> element) or CSS up to the closest ancestor block element after closing them at a <br> paragraph break. On the other hand, it does not reopen the directional embedding levels stemming from surrounding LRE/RLE/LRO/RLO and PDF characters.
Should the HTML specification of the bidi behavior of <br> be changed to this rather complicated definition in the hope that all browsers will be able to standardize around it?
And what about those rare uses of <br> when it is simply being used to wrap a line?
Proposed solution
The <br> situation can be resolved by the simple expedient of defining <br> as having bdi="yes" by default. (The bdi attribute is proposed in Section 2.1 .) Thus, it would not form a UBA paragraph break, but would be treated, by default, as if it had either LRM or RLM characters around it, providing the required bidi separation to fix the examples above. Not being a paragraph break, it would not close any explicit embedding levels surrounding it. The only visible difference compared to IE's current behavior should be when the <br> falls between an LRE, RLE, LRO, RLO and its matching PDF. Since the use of these characters around mark-up is both very rare and discouraged by the W3C, this exception is quite minor.
When the author wants to use <br> just to wrap a line without adding bidi separation, <br bdi="no"> will do the trick.
UTR #20 and UAX #13 will need to be updated to reflect this change. In the former, 'In HTML, use <xhtml:br /> instead of U+2028' should be replaced with 'In HTML, use <xhtml:br bdi="no" /> instead of U+2028'. In the latter, 'line separators basically correspond to HTML <BR>' should be replaced with 'line separators basically correspond to HTML <BR BDI="no">'.
Background
As in Section 3.1 .
The problem
IE and WebKit treat newline characters as a UBA paragraph break in <pre>, <textarea>, and the text displayed in dialogs by the page's scripts using functions such as Javascript's alert() and confirm(). Given that in these contexts newlines are expected to behave as they do in plain text, this would seem to be in accordance with the UBA. Firefox, however, treats newlines in all these contexts as UBA whitespace, while Opera treats them as UBA paragraph separators in <textarea> and dialog text, but as whitespace in <pre>. See Section 3.1 for examples where this makes a difference.
Proposed solution
The HTML specification should state that newline characters should be treated as UBA paragraph breaks in the following plain-text environments:
- In <textarea> (except immediately after the start tag or immediately preceding the end tag).
- Text passed by page scripts for display outside the page, via services such as Javascript's alert() and confirm() functions, or the proposed desktop notifications API
In <pre>, however, newlines can appear inside inline elements and thus have the same issues with being treated as UBA paragraph breaks as exist for <br>. Thus, a newline inside <pre> (except immediately after a start tag or immediately preceding an end tag, where the newline is ignored) should be treated as if it were a <br bdi="yes"> (see Section 3.1 ).
Where lines are wrapped automatically in any of these contexts, the wrapping should be treated as UBA whitespace.
Background
As in Section 3.1 .
The problem
There is no standard definition of whether a block element serves as a UBA break between the text preceding and following it, i.e. whether the text preceding a <div></div> or an <hr> (defined to be a block element) should behave as if it were in the same UBA paragraph as the text following it. For short, we will call block elements with text on both sides "embedded".
Different browsers treat embedded block elements differently. Just as with <br>, in Firefox and Opera, an embedded block element provides no bidi separation between the text preceding and following it, while IE and WebKit treat it as a UBA paragraph break. See Section 3.1 for examples where this discrepancy makes a difference; just replace <br> with <hr>.
It is difficult to justify Firefox and Opera's treatment of embedded block elements. Besides breaking a line, the embedded block elements:
- Include among them the paragraph element, <p>. It seems reasonable to expect the insertion of a paragraph to break the text before it and the text after it into two UBA paragraphs.
- The text before and after a block element is said to form "anonymous blocks", and it is well accepted that blocks should constitute UBA paragraphs.
- Unlike <br>, a block element can and usually does contain text of its own, which certainly forms a separate UBA paragraph. Having the text before an embedded block element affect the display of the text after it or vice-versa, skipping right over the text inside the embedded block element, is very strange, a quite unexpected "action at a distance".
Thus, it seems reasonable to resolve the discrepancy in favor of treating embedded elements as UBA paragraph breaks. Since inline elements are not allowed to contain block elements, we expect all ancestors of a block element to be block elements themselves, and thus do not expect the issues that we encountered with inline ancestors in the case of <br>.
Unfortunately, life is not simple, and the CSS "display" property allows making an inline element behave like a block element; this creature can then be legally placed inside a normal inline element. Making the inline-turned-to-block element act as a paragraph separator indeed creates a problem for its inline ancestors that create embedding levels (e.g. have dir attributes).
Furthermore, the CSS "display" property allows many more possibilities than good old "block" and "inline". Should these be treated as UBA paragraph breaks?
Proposed solution
Elements with block display should be specified as introducing a UBA paragraph break between the text preceding and following them.
If the block-display element has ancestors with inline display that have bidi properties (e.g. the dir attribute or the <bdo> element), these bidi properties should be applied to the anonymous block boxes created for these inline elements, in accordince with CSS specs for anonymous block boxes.
The text inside an elements with inline-block display should constitute a UBA paragraph, but probably should not introduce a UBA paragraph break. Instead, it can default to bdi="yes". This and the other types of display should be given more thorough investigation.
Background
The W3C recommends that in HTML, the direction of text be declared using the dir attribute, avoiding the use of Unicode formatting characters LRE, RLE, and PDF except where the dir attribute is inapplicable.
The problem
One would expect that the page's direction set using <html dir=...> or <body dir=...> would apply to the text displayed in dialogs by the page's scripts using functions such as Javascript's alert() and confirm(). This is in fact the case in IE. However, it is not the case in any other major browser. The directional context that the major non-IE browsers use for dialog text is either the OS or the browser chrome's default direction, which neither the server nor page scripts can even determine, let alone control.
Since a value displayed in the wrong direction can come out garbled, pages wind up having to wrap their RTL dialog text in RLE + PDF characters for correct display on LTR systems. On the other hand, pages dare not wrap their LTR dialog text in LRE + PDF characters for correct display on RTL systems, since most computers in the world are running an LTR OS without RTL script support turned on, and thus display LRE and PDF as rectangles. (This is not a concern in the case of RTL dialog text, since a system that does not have RTL script support will not display RTL text correctly anyway.) Furthermore, these formatting characters are little-known, lack named entities, and are generally undesirable in HTML documents.
Proposed solution
The HTML specification should state that text passed by page scripts for display outside the page, via services such as Javascript's alert() and confirm() functions, or the proposed desktop notifications API will be displayed in the <body> element's direction. Backward compatibility is not an issue here because this aspect of bidi behavior was never defined by the HTML standard, and browser behavior has not been consistent.
The proposed solution agrees with the dir on html, javascript alert box and dir on body, javascript alert box tests in the i18n test suite being developed by the W3C.
The local dir, javascript alert box test and similar ones for confirm() and prompt(), which instead assert that dialog text should be displayed in the triggering element's direction, should be modified to conform with the proposed solution. Making the dialog text direction dependent on the triggering element makes things too difficult for the page developer, since the same function when called for events triggered by different elements - or even the same element after its computed direction changes - will result in different dialog displays.
It is easy enough for a browser to implement the proposed solution, since it knows the default directional context in which the text will be displayed by the underlying platform. If and only if this differs from the page's direction, the browser needs to wrap (each paragraph of) the dialog text in RLE + PDF in an RTL page and LRE + PDF in an LTR page.
Background
As in Section 3.4 .
The problem
One would expect that the page's direction set using <html dir=...> would apply to the page's <title>. Unfortunately, however, this is not the case in any major browser. The directional context all major browsers use for <title> is either the OS or the browser chrome's default direction, which neither the server nor page scripts can even determine, let alone control.
Nor does setting the dir attribute directly on the <title> element have any effect in any major browser.
Since a value displayed using the wrong direction can come out garbled, pages wind up having to wrap their RTL <title> in RLE + PDF characters. This has the same problems as with script dialog text, see Section 3.4 .
Proposed solution
The HTML specification should explicitly state that the <title>'s text will be displayed in the <title>'s computed direction.
It is easy enough for a browser to implement this, since it knows the default directional context in which the text will be displayed. If and only if this differs from the desired direction, the browser needs to wrap the title text in RLE + PDF when RTL is desired and LRE + PDF when LTR is desired.
In principle, this could break existing RTL documents that count on their title being displayed in LTR, as is usually the case today. The change should be made despite this, because:
-
Such documents can't really count on the current behavior anyway: on an RTL OS / browser the title is already displayed RTL.
-
In many cases, RTL documents work around the problem by having a title that looks the same whether displayed in LTR or RTL.
-
This will fix more documents than it will break.
-
Forcing backward compatibility will perpetuate an ugly exception.
Background
As in Section 3.4 .
The problem
Currently all major browsers (IE, FF, Chrome, Safari, Opera) display the tooltips specified by a title or alt element attribute in the direction of the element to which it belongs, but this does not appear to be formally specified anywhere. Furthermore, this consensus seems fragile because in principle, the direction of an element and the text of its tooltip do not have to coincide. Here is a reasonable counterexample: an RTL web page displays an LTR address (e.g. for a location in Europe), with a tooltip on the address element saying "ADDRESS" in the page's language. The tooltip thus needs to be RTL while the element needs to be LTR.
Until recently, Chrome displayed tooltips in the OS / browser's default direction. When fixing this bug, the initial inclination was to apply only the page's direction, not the element's, due to the "in principle" consideration above.
Nevertheless, although counterexamples as given above can be found, tooltip text most usually does have the same direction as the element's text even where the element does have text, which is not very often. For such counterexamples, there is a simple workaround in the form of putting the tooltip on an extra element wrapping the original one.
Apparently not trusting browser behavior, the W3C suggests that tooltip direction may have to be set using LRE | RLE + PDF. This is actually quite difficult to do properly, since wrapping an LTR tooltip in LRE + PDF just in case the browser winds up displaying it in an RTL context will result in the LRE and PDF displaying as rectangles on LTR OS's without RTL support enabled, i.e. the vast majority of computers.
Proposed solution
The HTML specification should explicitly state that title and alt attribute text will be displayed in the element's computed direction.
Background
As in Section 3.4 .
The problem
In a single <select>, the values of different options may have different directions. Currently, however, out of all major browsers, only FF supports the dir attribute on <option>, and does so poorly: once the value is chosen, it is displayed in the <select>'s direction.
IE and Opera display all options in the <select>'s direction.
Safari automatically estimates the direction of each option and displays it as such both in the dropdown and after it has been chosen regardless of the <select>'s direction (which is only used to place the down-arrow button and to align the values). This is all very nice, but direction estimation algorithms do make mistakes, so it would be good to be able to specify the actual dir value for a given <option> - and Safari does not support that.
Chrome does not support the dir attribute on <option> and is on its way to doing what Safari does.
As a result, the only practical way to specify <option> value direction is using LRE | RLE + PDF, which is cumbersome.
Proposed solution
The HTML specification should state that an <option> element's computed direction will take its dir attribute into account, and will be used to display the option's text in both the dropdown and after being chosen.
The HTML specification should also state that setting an <option> element's alignment via CSS or the align attribute will affect its display accordingly in both the dropdown and after being chosen.
Background
Garbling by incorrect direction also applies to text being entered by the user in an input control. In fact, entering text of direction opposite to the input's declared direction is an unpleasant experience even if the full text does not wind up being garbled, due to the cursor and punctuation jumping around during data entry and difficulty in selecting text. All major browsers thus provide some way for the user to set the direction of each <input type="text"> and <textarea> element.
The problem
Unfortunately, the way "set direction" functionality interacts with page scripts varies significantly between browsers, which makes it difficult to write scripts that are informed of the user's choice.
IE: Direction is set using keyboard shortcuts - CTRL + LEFT SHIFT for LTR and CTRL + RIGHT SHIFT for RTL. (These key combinations are also adopted for this purpose by most Microsoft products, e.g. Windows dialogs, notepad and Word.) They set the value of the element's dir attribute, which is then available to scripts. They trigger the onpropertychange event, at which time the dir value is already changed. They also trigger onkeyup, but before the dir value has been changed, so setTimeout(0) has to be used to get the updated dir value. They do not trigger onkeypress.
FF: Direction is set using the CTRL + SHIFT + X keyboard shortcut, which cycles through LTR and RTL. It does not set the value of the element's dir attribute, and is thus invisible to scripts.
Opera: same keyboard shortcuts as IE. They do not set the value of the element's dir attribute, and are thus invisible to scripts.
Chrome: same keyboard shortcuts as IE. They set the value of the element's dir attribute, which is then available to scripts. They trigger the onkeyup event, at which time the dir value is already changed. They do not trigger onkeypress or oninput. They also do not trigger onpropertychange, since this event exists only in IE.
Safari: Right-click on the <input> or <textarea> provides a "Set paragraph direction" submenu. Using "Set paragraph direction" sets the value of the element's dir attribute, which is then available to scripts. However, it does not trigger onkeyup, onkeypress, or oninput. It also doesn't trigger onpropertychange, since this event exists only in IE.
Proposed solution
The HTML specification should state that some way to set the direction of <input type="text"> and <textarea> elements should be exposed to the user, and using it will:
- Set the element's dir attribute value accordingly.
- Trigger oninput after the dir attribute has been set; even though no actual input took place, the user did change the recommended interpretation of the input already collected.
Furthermore, it should be recommended that on an OS that has a widespread convention for setting direction (such as CTRL + LEFT SHIFT for LTR and CTRL + RIGHT SHIFT for RTL on Windows), the user agent will support that convention (although it may provide other methods too).
Some browsers implement auto-completion, a feature whereby values previously entered into an element like <input type="text"> are remembered and under certain conditions presented to the user in a dropdown. When the user selects one of the items in the dropdown, this value is assigned to the element. At different times, the user may enter values of different direction for the same input. The direction of a value is set either directly by the user through a "set direction" command exposed by the browser (e.g. via keyboard shortcuts, see Section 3.8 ) or letting page scripts automatically set the input's dir attribute after estimating the direction of the value on the fly.
Browsers do not remember the direction of previously-entered values. Some display them in the dropdown in the OS or browser default direction. Some display them in the input's current direction. Finally, some display each value in its own estimated direction. Each of these will result in some values being displayed incorrectly; even the last approach will sometimes fail because estimation algorithms do make mistakes, and this may not have been the direction originally set by the user or page scripts.
After the user chooses a value from the dropdown, the value is usually displayed in the input's current direction, which may or may not be correct for it.
The HTML specification should state that whenever a user agent stores a user-provided <input type="text"> or <textarea> value for later use (such as auto-completion), it should also store the nominal direction value the element had when displaying this value. This may be the original direction of the element, or may have been set by the user for that value via keyboard shortcuts, or may have been set for that value by page scripts. If the user agent later displays the value in an auto-completion dropdown, it should be displayed in its stored direction. If the value is assigned to an element, the element's dir value should be set to its stored direction.
Background
The HTML specifications gives no indication of how the bullet or number of an <li> element should be displayed when its computed direction is the opposite of its parent's (usually an <ol> or <ul>).
The problem
In practice, for <li> elements whose "list-style-position" CSS property has the default "outside" value, different browsers do different things. Furthermore, the effects vary depending on the list's alignment, and whether it is ordered or unordered:
<ul dir="ltr"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
| IE | Firefox, Opera and WebKit |
|---|
|
* item a.
* longer item b.
.C METI LTR *
|
* item a.
* longer item b.
.C METI LTR
|
<ol dir="ltr"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
| IE | Firefox, Opera and WebKit |
|---|
|
1. item a.
2. longer item b.
.C METI LTR .
|
1. item a.
2. longer item b.
.C METI LTR
|
<ul dir="ltr" style="text-align:left"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
| IE | Firefox and Opera | WebKit |
|---|
|
* item a.
* longer item b.
.C METI LTR *
|
* item a.
* longer item b.
.C METI LTR *
|
* item a.
* longer item b.
.C METI LTR
|
<ul dir="ltr" style="text-align:right"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
| IE | Firefox and Opera | WebKit |
|---|
|
* item a.
* longer item b.
.C METI LTR *
|
* item a.
* longer item b.
.C METI LTR
|
*
item a.
*
longer item b.
.C METI LTR
|
In our opinion, not only is browser behavior unacceptably incompatible and inconsistent, but none of the above provides a usable display of opposite-direction list items.
Proposed solution
In our opinion, the bullets/numbers of all "list-style-position:outside" items, regardless of their direction, should always:
This can be easily achieved by specifying that the rendering of "list-style-position:outside" list item bullets should depend on the direction of the list element (<ul>, <ol>, etc.), and not depend on the direction of the individual list items or the alignment of either the list or the list item. The outcome should look like this:
<ul dir="ltr"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
|
* item a.
* longer item b.
|
<ol dir="ltr"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
|
1. item a.
2. longer item b.
|
<ul dir="ltr" style="text-align:left"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
|
* item a.
* longer item b.
* .C METI LTR
|
<ul dir="ltr" style="text-align:right"><li>item a.</li><li> longer item b.</li><li dir="rtl">RTL ITEM C.</li></ul>
|
*
item a.
*
longer item b.
*
.C METI LTR
|
The HTML specification should also state that setting an <li> element's alignment via CSS or the align attribute will affect its display accordingly.
The vertical scrollbar in an LTR UIs is normally placed on the right side of a window or widget, and on the left side in an RTL UI.
In a browser open on a given page, the UI is made up of two parts: the chrome of the browser itself (e.g. its menus and toolbars), and the page being displayed in the browser. The two parts can be and often are in two different langauges and thus directions.
It is unclear which of the two is the principal part of the UI. Certainly the page takes up most of the window and is presumably the user's focus of attention. As a result, it seems natural that the vertical scrollbar should be on the "end" edge relative to the page's (i.e. the <body> element's) overall direction - and not the browser's chrome direction.
However, this usually results in a usability issue when surfing: the scrollbar moves from side to side when going from an LTR page to an RTL page or vice-versa, confusing the user and making the scrollbar surprisingly difficult to find visually and click on physically. It is also arguable that the overall scrollbar is a part of the browser chrome, not the page, so it has no business being dependent on the page direction.
As a result, Firefox, Chrome and Safari place the scrollbar on the "end" edge relative to the browser's chrome direction. Furthermore, this is the behavior required by the dir on html, vertical scrollbar alignment and dir on body, vertical scrollbar alignment test tests in the i18n test suite being developed by the W3C.
However, IE and Opera continue to put the scrollbar relative to the page direction.
The HTML specification should state that the user agent window's overall vertical scrollbar should be located independent of the direction of any page element, despite being otherwise controlled by the style of the <body> element. (Thus, it should be located on the the "end" side relative to the user agent chrome direction.)
Background
As in Section 3.1 .
The problem
Users expect the vertical scrollbar of a "widget" inside the page to be on an LTR widget's right side, and on an RTL widget's left side. The rationale for making the browser chrome direction determine the location of the vertical scrollbar for the <body> element in Section 3.11 were exceptional to the <body> element:
However, due to the usability problem with the page's overall vertical scrollbar described in Section 3.11 , Firefox, Chrome and Safari place every element's vertical scrollbar on the "end" edge relative to the browser's chrome direction, regardless of the element's direction. While this is indeed desirable for the <body> element as indicated in Section 3.11 , it is not desirable for the elements below it.
Proposed solution
The HTML specification should state that the vertical scrollbar of an element below <body> should be on the "end" side relative to the element's direction.
