is_language_tag("fr") is TRUE
is_language_tag("x-jicarilla") is FALSE
(Subtags can be 8 chars long at most -- jicarilla is 9)
is_language_tag("sgn-US") is TRUE
(Thats American Sign Language)
is_language_tag("i-Klikitat") is TRUE
(True without regard to the fact noone has actually
registered Klikitat -- its a formally valid tag)
is_language_tag("fr-patois") is TRUE
(Formally valid -- altho descriptively weak!)
is_language_tag("Spanish") is FALSE
is_language_tag("french-patois") is FALSE
(No good -- first subtag has to match
/^([xXiI]|[a-zA-Z]{2,3})$/ -- see RFC3066)
is_language_tag("x-borg-prot2532") is TRUE
(Yes, subtags can contain digits, as of RFC3066)

o

the function extract_language_tags($whatever)

Returns a list of whatever looks like formally valid language tags
in $whatever. Not very smart, so dont get too creative with
what you want to feed it.

same_language_tag(x-kadara, i-kadara) is TRUE
(The x/i- alternation doesnt matter)
same_language_tag(X-KADARA, i-kadara) is TRUE
(...and neither does case)
same_language_tag(en, en-US) is FALSE
(all-English is not the SAME as US English)
same_language_tag(x-kadara, x-kadar) is FALSE
(these are totally unrelated tags)
same_language_tag(no-bok, nb) is TRUE
(no-bok is a legacy tag for nb (Norwegian Bokmal))

same_language_tag works by just seeing whether
encode_language_tag($lang1) is the same as
encode_language_tag($lang2).

(Yes, I know this function is named a bit oddly. Call it historic
reasons.)

o

the function similarity_language_tag($lang1, $lang2)

Returns an integer representing the degree of similarity between
tags $lang1 and $lang2 (the order of which does not matter), where
similarity is the number of common elements on the left,
without regard to case and to x/i- alternation.

is_dialect_of(en-US, en) is TRUE
(American English IS a dialect of all-English)
is_dialect_of(fr-CA-joual, fr-CA) is TRUE
is_dialect_of(fr-CA-joual, fr) is TRUE
(Joual is a dialect of (a dialect of) French)
is_dialect_of(en, en-US) is FALSE
(all-English is a NOT dialect of American English)
is_dialect_of(fr, en-CA) is FALSE
is_dialect_of(en, en ) is TRUE
is_dialect_of(en-US, en-US) is TRUE
(B<Note:> these are degenerate cases)
is_dialect_of(i-mingo-tom, x-Mingo) is TRUE
(the x/i thing doesnt matter, nor does case)
is_dialect_of(nn, no) is TRUE
(because nn (New Norse) is aliased to no-nyn,
as a special legacy case, and no-nyn is a
subform of no (Norwegian))

o

the function super_languages($lang1)

Returns a list of language tags that are superordinate tags to $lang1
 it gets this by removing subtags from the end of $lang1 until
nothing (or just i or x) is left.

super_languages("fr-CA-joual") is ("fr-CA", "fr")
super_languages("en-AU") is ("en")
super_languages("en") is empty-list, ()
super_languages("i-cherokee") is empty-list, ()
...not ("i"), which would be illegal as well as pointless.

If $lang1 is not a valid language tag, returns empty-list in
a list context, undef in a scalar context.

A notable and rather unavoidable problem with this method:
x-mingo-tom has an x because the whole tag isnt an
IANA-registered tag  but super_languages(x-mingo-tom) is
(x-mingo)  which isnt really right, since i-mingo is
registered. But this module has no way of knowing that. (But note
that same_language_tag(x-mingo, i-mingo) is TRUE.)

More importantly, you assume at your peril that superordinates of
$lang1 are mutually intelligible with $lang1. Consider this
carefully.

o

the function locale2language_tag($locale_identifier)

This takes a locale name (like en, en_US, or en_US.ISO8859-1)
and maps it to a language tag. If its not mappable (as with,
notably, C and POSIX), this returns empty-list in a list context,
or undef in a scalar context.

locale2language_tag("en") is "en"
locale2language_tag("en_US") is "en-US"
locale2language_tag("en_US.ISO8859-1") is "en-US"
locale2language_tag("C") is undef or ()
locale2language_tag("POSIX") is undef or ()
locale2language_tag("POSIX") is undef or ()

Im not totally sure that locale names map satisfactorily to language
tags. Think REAL hard about how you use this. YOU HAVE BEEN WARNED.

The output is untainted. If you dont know what tainting is,
dont worry about it.

o

the function encode_language_tag($lang1)

This function, if given a language tag, returns an encoding of it such
that:

* an encoding of a formally valid language tag always is a string
value that is defined, has length, and is trueif considered as a
boolean.

Note that the encoding itself is <B>notB> a formally valid language tag.
Note also that you cannot, currently, go from an encoding back to a
language tag that its an encoding of.

Note also that you <B>mustB> consider the encoded value as atomic; i.e.,
you should not consider it as anything but an opaque, unanalysable
string value. (The internals of the encoding method may change in
future versions, as the language tagging standard changes over time.)

encode_language_tag returns undef if given anything other than a
formally valid language tag.

The reason encode_language_tag exists is because different language
tags may represent the same language; this is normally treatable with
same_language_tag, but consider this situation:

You have a data file that expresses greetings in different languages.
Its format is [language tag]=[how to say Hello], like:

en-US=Hiho
fr=Bonjour
i-mingo=Hau

And suppose you write a program that reads that file and then runs as
a daemon, answering client requests that specify a language tag and
then expect the string that says how to greet in that language. So an
interaction looks like:

and then just answer client requests for language $wanted by just
looking up

$greetings{encode_language_tag($wanted)}

And that does the Right Thing.

o

the function alternate_language_tags($lang1)

This function, if given a language tag, returns all language tags that
are alternate forms of this language tag. (I.e., tags which refer to
the same language.) This is meant to handle legacy tags caused by
the minor changes in language tag standards over the years; and
the x-/i- alternation is also dealt with.

Note that this function does not try to equate new (and never-used,
and unusable)
ISO639-2 three-letter tags to old (and still in use) ISO639-1
two-letter equivalents  like ara -> ar  because
ara has never been in use as an Internet language tag,
and RFC 3066 stipulates that it never should be, since a shorter
tag (ar) exists.

Examples:

alternate_language_tags(no-bok) is (nb)
alternate_language_tags(nb) is (no-bok)
alternate_language_tags(he) is (iw)
alternate_language_tags(iw) is (he)
alternate_language_tags(i-hakka) is (zh-hakka, x-hakka)
alternate_language_tags(zh-hakka) is (i-hakka, x-hakka)
alternate_language_tags(en) is ()
alternate_language_tags(x-mingo-tom) is (i-mingo-tom)
alternate_language_tags(x-klikitat) is (i-klikitat)
alternate_language_tags(i-klikitat) is (x-klikitat)

This function returns empty-list if given anything other than a formally
valid language tag.

o

the function @langs = panic_languages(@accept_languages)

This function takes a list of 0 or more language
tags that constitute a given users Accept-Language list, and
returns a list of tags for other (non-super)
languages that are probably acceptable to the user, to be
used if all else fails.

For example, if a user accepts only ca (Catalan) and
es (Spanish), and the documents/interfaces you have
available are just in German, Italian, and Chinese, then
the user will most likely want the Italian one (and not
the Chinese or German one!), instead of getting
nothing. So panic_languages(ca, es) returns
a list containing it (Italian).

English (en) is always in the return list, but
whether its at the very end or not depends
on the input languages. This function works by consulting
an internal table that stipulates what common
languages are close to each other.

This takes a list of strings (which are presumed to be language-tags;
strings that arent, are ignored); and after each one, this function
inserts super-ordinate forms that dont already appear in the list.
The original list, plus these insertions, is returned.

This works like implicate_supers except that the implicated
forms are added to the end of the return list.

In other words, implicate_supers_strictly takes a list of strings
(which are presumed to be language-tags; strings that arent, are
ignored) and after the whole given list, it inserts the super-ordinate forms
of all given tags, minus any tags that already appear in the input list.

In other words, it takes this:

pt-br de-DE en-US fr pt-br-janeiro

and returns this:

pt-br de-DE en-US fr pt-br-janeiro pt de en

The reason this function has _strictly in its name is that when
youre processing an Accept-Language list according to the RFCs, if
you interpret the RFCs quite strictly, then you would use
implicate_supers_strictly, but for normal use (i.e., common-sense use,
as far as Im concerned) youd use implicate_supers.

Ive considered making all the above functions that output language
tags return all those tags strictly in lowercase. Having all your
language tags in lowercase does make some things easier. But you
might as well just lowercase as you like, or call
encode_language_tag($lang1) where appropriate.

This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.

The programs and documentation in this dist are distributed in
the hope that they will be useful, but without any warranty; without
even the implied warranty of merchantability or fitness for a
particular purpose.