Malayalam/Brief History of Malayalam Chillu Encoding

Between around 1999 and 2002, the Indian government did research to identify what they thought was lacking in Unicode for Indic scripts. They gave their first feedback in 2000; most of their results were published by TDIL in 2002. They expected UTC to take action accordingly, though it appears they did not have a good understanding of UTC and ISO procedures at the time, and they did not submit specific proposals. Without specific proposal, UTC took no action.

In 2004, partly due to awareness on the part of UTC members that there were outstanding Indic issues and partly as a result of the head of the Indian Minstrity of IT engaging with UTC to request that UTC consider the changes India had requested, UTC took action to examine what issues remained for them to consider.

One of the changes requested by India was to encode atomic chillu characters. A representation for chillu characters was already specified in TUS4, and in such cases UTC takes a conservative approach: don't encode something new unless there's reason to believe that what is there already isn't fully adequate. Since the Indian government (a Consortium member) had requested action, UTC issued a Public Review item (PR 66) requesting input on the question of encoding atomic chillu characters. Also, the topic was raised on the Indic list.

Responses submitted generally supported encoding of the chillu characters. The feedback included data apparently supporting the proposal, as well as input from people who had seriously attempted to grapple with the encoding and text processing issues. Since this corroborated the request of the Indian government, it seemed appropriate to encode these characters. This was done at the UTC meeting in May 2005.

Just prior to the next UTC meeting, Rachana Aksharavedi submitted their long documents presenting arguments against encoding. As I recall, this was the first strong opposition to encoding the atomic chillu characters. This indicated to UTC that the responses to PR 66 were not representative of the entire user community and may not fully considered all the arguments, and so UTC took the extraordinary action of rescinding their previous decision so as to allow further investigation and evaluation.

Naturally, this led to some significant responses. If anything, discussion on the Indic list became more heated. And the Indian government expressed their concern that their requests were being further delayed, putting off further what was still needed (in their understanding) to deliver IT solutions for their languages. (Keep in mind they had first voiced their requests five years earlier.)

Earlier this year, after heroically attempting to sift though all the documents and email, Eric Muller prepared a document summarizing the issues and the significant arguments in favour of encoding -- these constituted issues that (assuming the data was factual) the current encoded representation did not seem to handle well. On that basis, and with the Indian governments on-going request, UTC agreed once again to encode these characters (May 2006).

At the same time, UTC was also aware that some Malayalees remained opposed to encoding these characters, and that some of these had indicated they were going to work with other stakeholders in India, including the government, towards arriving at a consensus. Hence, UTC once more took action to delay the encoding process by having the characters moved, within the ISO process, from amendment 3 of ISO 10646 to amendment 4.