
=head1 NAME

ShiftJIS::String - functions to manipulate Shift_JIS strings

=head1 SYNOPSIS

  use ShiftJIS::String;

  ShiftJIS::String::substr($str, ShiftJIS::String::index($str, $substr));

=head1 ABOUT THIS POD

This POD is written in Shift_JIS.

Do you see 'C<あ>' as C<HIRAGANA LETTER A>?
or 'C<\>' as C<YEN SIGN>, not as C<REVERSE SOLIDUS>?
Otherwise you'd change your font to an appropriate one.
(or the POD might be badly converted.)

=head1 DESCRIPTION

This module provides some functions which emulate
the corresponding C<CORE> functions and helps someone
to manipulate multiple-byte character sequences in Shift_JIS.

* 'Hankaku' and 'Zenkaku' mean 'halfwidth' and 'fullwidth' characters
in Japanese, respectively.

=head1 FUNCTIONS

=head2 Check Whether the String is Legal

=over 4

=item C<issjis(LIST)>

Returns a boolean indicating whether all the strings in the parameter list
are legally coded in Shift_JIS.

=back

=head2 Length

=over 4

=item C<length(STRING)>

Returns the length in characters of the supplied string.

=back

=head2 Reverse

=over 4

=item C<strrev(STRING)>

Returns a reversed string (having all characters in the opposite order).

=back

=head2 Search

=over 4

=item C<index(STRING, SUBSTR)>

=item C<index(STRING, SUBSTR, POSITION)>

Returns the position of the first occurrence
of C<SUBSTR> in C<STRING> at or after C<POSITION>.
If C<POSITION> is omitted, starts searching
from the beginning of the string.

If the substring is not found, returns -1.

=item C<rindex(STRING, SUBSTR)>

=item C<rindex(STRING, SUBSTR, POSITION)>

Returns the position of the last occurrence of C<SUBSTR> in C<STRING>.
If C<POSITION> is specified, returns the last
occurrence at or before that position.

If the substring is not found, returns -1.

=item C<strspn(STRING, SEARCHLIST)>

Returns returns the position of the first occurrence of
any character not contained in the search list.

  strspn("+0.12345*12", "+-.0123456789");
  # returns 8.

If the specified string does not contain any character
in the search list, returns 0.

The string consists of characters in the search list,
the returned value equals the length of the string.

=item C<strcspn(STRING, SEARCHLIST)>

Returns returns the position of the first occurrence of
any character contained in the search list.

  strcspn("Perlは面白い。", "赤青黄白黒");
  # returns 6.

If the specified string does not contain any character
in the search list,
the returned value equals the length of the string.

=back

=head2 Substring

=over 4

=item C<substr(STRING or SCALAR REF, OFFSET)>

=item C<substr(STRING or SCALAR REF, OFFSET, LENGTH)>

=item C<substr(SCALAR, OFFSET, LENGTH, REPLACEMENT)>

It works like C<CORE::substr>, but
using character semantics of Shift_JIS.

If the C<REPLACEMENT> as the fourth parameter is specified, replaces
parts of the C<SCALAR> and returns what was there before.

You can utilize the lvalue reference,
returned if a reference to a scalar variable is used as the first argument.

    ${ &substr(\$str,$off,$len) } = $replace;

        works like

    CORE::substr($str,$off,$len) = $replace;

The returned lvalue is not Shift_JIS-oriented but byte-oriented,
then successive assignment may cause unexpected results.

    $str = "0123456789";
    $lval  = &substr(\$str,3,1);
    $$lval = "あい";
    $$lval = "a";
    # $str is NOT "012aい456789", but an illegal string "012a\xA0い456789".

=back

=head2 Split

=over 4

=item C<strsplit(SEPARATOR, STRING)>

=item C<strsplit(SEPARATOR, STRING, LIMIT)>

This function emulates C<CORE::split>, but splits on the C<SEPARATOR> string,
not by a pattern.
If not in list context, only return the number of fields found,
but does not split into the C<@_> array.

  strsplit('||', '||あいうえお||パピプペポ||01234||');
  # ('', 'あいうえお', 'パピプペポ', '01234')

  strsplit('／', 'Perl／駱駝／Camel');
  # ('Perl', '駱駝', 'Camel')

If an empty string is specified as C<SEPARATOR>, splits the specified string
into characters (similarly to C<CORE::split //, STRING, LIMIT>).

  strsplit('', 'This is Perl.', 7);
  # ('T', 'h', 'i', 's', ' ', 'i',  's Perl.')

If an undefined value is specified as C<SEPARATOR>, splits the specified string
on whitespace characters (including C<IDEOGRAPHIC SPACE>).
Leading whitespace characters do not produce any field
(similarly to C<CORE::split ' ', STRING, LIMIT>).

  strsplit(undef, ' 　 This  is 　 Perl.');
  # ('This', 'is', 'Perl.')

=back

=head2 Comparison

=over 4

=item C<strcmp(LEFT-STRING, RIGHT-STRING)>

Returns 1 (when C<LEFT-STRING> is greater than C<RIGHT-STRING>)
or 0 (when C<LEFT-STRING> is equal to C<RIGHT-STRING>)
or -1 (when C<LEFT-STRING> is lesser than C<RIGHT-STRING>).

The order is roughly as shown the following list.

    JIS X 0201 Roman, JIS X 0201 Kana, then JIS X 0208 Kanji (Zenkaku).

For example,
C<0x41> as C<'A'> is lesser than C<0xB1> (C<'ｱ' HANKAKU KATAKANA A>).
C<0xB1> as C<'ｱ'> is lesser than C<0x8341> (C<'ア' KATAKANA A>).
C<0x8341> as C<'ア'> is lesser than C<0x8383> (C<'ャ' KATAKANA SMALL YA>).
C<0x8383> as C<'ャ'> is lesser than C<0x83B1> (C<'Τ' GREEK CAPITAL TAU>).

B<Caveat!>
Compare the 2nd and the 4th examples.
Byte C<"\xB1"> is lesser than byte C<"\x83"> as the leading bytes;
while greater as the trailing bytes.
Shortly, the ordering as binary is broken for the Shift_JIS codepoint order.

=item C<strEQ(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is equal to C<RIGHT-STRING>.

B<Note:> C<strEQ> is an expensive equivalence of the C<CORE>'s C<eq> operator.

=item C<strNE(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is not equal to C<RIGHT-STRING>.

B<Note:> C<strNE> is an expensive equivalence of the C<CORE>'s C<ne> operator.

=item C<strLT(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is lesser than C<RIGHT-STRING>.

=item C<strLE(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is lesser than or equal to C<RIGHT-STRING>.

=item C<strGT(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is greater than C<RIGHT-STRING>.

=item C<strGE(LEFT-STRING, RIGHT-STRING)>

Returns a boolean whether C<LEFT-STRING> is greater than or equal to C<RIGHT-STRING>.

=item C<strxfrm(STRING)>

Returns a string transformed so that C<CORE:: cmp> can be used
for binary comparisons (B<NOT> the length of the transformed string).

I.e. C<strxfrm($a) cmp strxfrm($b)> is equivalent to C<strcmp($a, $b)>,
as long as your C<cmp> doesn't use any locale other than that of Perl.

=back

=head2 Character Range

=over 4

=item C<mkrange(EXPR, EXPR)>

Returns the character list (not in list context, as a concatenated string)
gained by parsing the specified character range.

A character range is specified with a C<HYPHEN-MINUS>, C<'-'>. The backslashed
combinations C<'\-'> and C<'\\'> are used instead of the characters
C<'-'> and C<'\'>, respectively. The hyphen at the beginning or
end of the range is also evaluated as the hyphen itself.

For example, C<mkrange('+\-0-9a-fA-F')> returns
C<('+', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9',
'a', 'b', 'c', 'd', 'e', 'f', 'A', 'B', 'C', 'D', 'E', 'F')>
and C<scalar mkrange('か-ご')> returns C<'かがきぎくぐけげこご'>.

The order of Shift_JIS characters is:
  C<0x00 .. 0x7F, 0xA1 .. 0xDF, 0x8140 .. 0x9FFC, 0xE040 .. 0xFCFC>.
So, mkrange('亜-腕') returns the list of all characters in level 1 Kanji.

If true value is specified as the second parameter,
Reverse character ranges such as C<'9-0'>, C<'Z-A'> can be used;
otherwise, reverse character ranges are croaked.

=back

=head2 Transliteration

=over 4

=item C<strtr(STRING or SCALAR REF, SEARCHLIST, REPLACEMENTLIST)>

=item C<strtr(STRING or SCALAR REF, SEARCHLIST, REPLACEMENTLIST, MODIFIER)>

=item C<strtr(STRING or SCALAR REF, SEARCHLIST, REPLACEMENTLIST, MODIFIER, PATTERN)>

=item C<strtr(STRING or SCALAR REF, SEARCHLIST, REPLACEMENTLIST, MODIFIER, PATTERN, TOPATTERN)>

Transliterates all occurrences of the characters found in the search list
with the corresponding character in the replacement list.

If a reference to a scalar variable is specified as the first argument,
returns the number of characters replaced or deleted;
otherwise, returns the transliterated string and
the specified string is unaffected.

  $str = "なんといおうか";
  print strtr(\$str,"あいうえお", "アイウエオ"), "  ", $str;
  # output: 3  なんとイオウか

  $str = "後門の狼。";
  print strtr($str,"後狼。", "前虎、"), $str;
  # output: 前門の虎、後門の狼。

B<SEARCHLIST and REPLACEMENTLIST>

Character ranges such as C<"ぁ-お"> (internally utilizing C<mkrange()>)
are supported.

If the C<REPLACEMENTLIST> is empty, the C<SEARCHLIST> is replicated.

If the replacement list is shorter than the search list,
the final character in the replacement list
is replicated till it is long enough
(but differently works when the 'd' modifier is used).

  strtr(\$str, 'ぁ-んァ-ヶｦ-ﾟ', '#');
    # replaces all Kana letters by '#'.

B<MODIFIER>

    c   Complement the SEARCHLIST.
    d   Delete found but unreplaced characters.
    s   Squash duplicate replaced characters.
    h   Returns a hash (or a hashref in scalar context) of histogram
    R   No use of character ranges.
    r   Allows to use reverse character ranges.
    o   Caches the conversion table internally.

  strtr(\$str, 'ぁ-んァ-ヶｦ-ﾟ', '');
    # counts all Kana letters in $str.

  $onlykana = strtr($str, 'ぁ-んァ-ヶｦ-ﾟ', '', 'cd');
    # deletes all characters except Kana letters.

  strtr(\$str, " \x81\x40\n\r\t\f", '', 'd');
    # deletes all whitespace characters including IDEOGRAPHIC SPACE.

  strtr("おかかうめぼし　ちちとはは", 'ぁ-ん', '', 's');
    # output: おかうめぼし　ちとは

  strtr("条件演算子の使いすぎは見苦しい", 'ぁ-ん', '＃', 'cs');
    # output: ＃の＃いすぎは＃しい

If C<'h'> modifier is specified, returns a hash
(or a hashref in scalar context) of histogram,
whether the first argument is a reference or not.
If you want to get the histogram and the modified string at once,
pass a reference as the first argument and use its value after.

  %hist_kana = strtr('日本語のカタカナ', 'ぁ-んァ-ヶｦ-ﾟ', '', 'h');
    # examine apperence frequency of kana
    # returns ('の' => 1, 'カ' => 2, 'タ' => 1, 'ナ' => 1),
    # but the key order may differ from this.

If C<'R'> modifier is specified, C<'-'> is not evaluated as a meta character
but C<HYPHEN-MINUS> itself like in C<tr'''>. Compare:

  strtr("90 - 32 = 58", "0-9", "A-J");
    # output: "JA - DC = FI"

  strtr("90 - 32 = 58", "0-9", "A-J", "R");
    # output: "JA - 32 = 58"
    # cf. ($str = "90 - 32 = 58") =~ tr'0-9'A-J';
    # '0' to 'A', '-' to '-', and '9' to 'J'.

If C<'r'> modifier is specified, you are allowed to use reverse
character ranges. For example, C<strtr($str, "0-9", "9-0", "r")>
is equivalent to C<strtr($str, "0123456789", "9876543210")>.

  strtr($text, '亜-腕', '腕-亜', "r");
    # Your text may seem to be clobbered.

B<PATTERN and TOPATTERN>

By use of C<PATTERN> and C<TOPATTERN>, you can transliterate the string
using lists containing some multi-character substrings.

If called with four arguments, C<SEARCHLIST>, C<REPLACEMENTLIST>,
and C<STRING> are splited characterwise;

If called with five arguments, a multi-character substring
that matchs C<PATTERN> in C<SEARCHLIST>, C<REPLACEMENTLIST>, or C<STRING>
is regarded as an transliteration unit.

If both C<PATTERN> and C<TOPATTERN> are specified,
a multi-character substring
either that matchs C<PATTERN> in C<SEARCHLIST>, or C<STRING>,
or that matchs C<TOPATTERN> in C<REPLACEMENTLIST>
is regarded as an transliteration unit.

  print strtr(
    "Caesar Aether Goethe",
    "aeoeueAeOeUe",
    "&auml;&ouml;&ouml;&Auml;&Ouml;&Uuml;",
    "",
    "[aouAOU]e",
    "&[aouAOU]uml;");

  # output: C&auml;sar &Auml;ther G&ouml;the

B<LISTs as Anonymous Arrays>

Instead of specification of C<PATTERN> and C<TOPATTERN>, you can use
anonymous arrays as C<SEARCHLIST> and/or C<REPLACEMENTLIST> as follows.

  print strtr(
    "Caesar Aether Goethe",
    [qw/ae oe ue Ae Oe Ue/],
    [qw/&auml; &ouml; &ouml; &Auml; &Ouml; &Uuml;/]
  );

B<Caching the conversion table>

If C<'o'> modifier is specified, the conversion table is cached internally.
e.g.

  foreach (@hiragana_strings) {
    print strtr($_, 'ぁ-ん', 'ァ-ン', 'o');
  }
  # katakana strings are printed

will be almost as efficient as this:

  $hiragana_to_katakana = trclosure('ぁ-ん', 'ァ-ン');

  foreach (@hiragana_strings) {
    print &$hiragana_to_katakana($_);
  }

You can use whichever you like.

Without C<'o'>,

  foreach (@hiragana_strings) {
    print strtr($_, 'ぁ-ん', 'ァ-ン');
  }

will be very slow since the conversion table is made
whenever the function is called.

=back

=head2 Generation of the Closure to Transliterate

=over 4

=item C<trclosure(SEARCHLIST, REPLACEMENTLIST)>

=item C<trclosure(SEARCHLIST, REPLACEMENTLIST, MODIFIER)>

=item C<trclosure(SEARCHLIST, REPLACEMENTLIST, MODIFIER, PATTERN)>

=item C<trclosure(SEARCHLIST, REPLACEMENTLIST, MODIFIER, PATTERN, TOPATTERN)>

Returns a closure to transliterate the specified string.
The return value is an only code reference, not blessed object.
By use of this code ref, you can save yourself time
as you need not specify the parameter list every time.

  my $digit_tr = trclosure("1234567890-", "一二三四五六七八九〇−");
  print &$digit_tr ("TEL ：0124-45-6789\n"); # ok to perl 5.003
  print $digit_tr->("FAX ：0124-51-5368\n"); # perl 5.004 or better

  # output:
  # TEL ：〇一二四−四五−六七八九
  # FAX ：〇一二四−五一−五三六八

The functionality of the closure made by C<trclosure()> is equivalent
to that of C<strtr()>. Frankly speaking, the C<strtr()> calls
C<trclosure()> internally and uses the returned closure.

=back

=head2 Case of the Alphabet

=over 4

=item C<toupper(STRING or SCALAR REF)>

Returns an uppercased string of C<STRING>.
Converts only half-width Latin characters C<a-z> to C<A-Z>.

If a reference of scalar variable is specified as the first argument,
the string referred to it is uppercased and
the number of characters replaced is returned.

=item C<tolower(STRING or SCALAR REF)>

Returns a lowercased string of C<STRING>.
Converts only half-width Latin characters C<A-Z> to C<a-z>.

If a reference of scalar variable is specified as the first argument,
the string referred to it is lowercased and
the number of characters replaced is returned.

=back

=head2 Conversion between hiragana and katakana

If a reference to a scalar variable is specified as the first argument,
the string referred to it is converted and
the number of characters replaced is returned.
Otherwise, returns a string converted and
the specified string is unaffected.

B<Note:> The conversion between a voiced (or semivoiced)
katakana (or hiragana), such as C<'ガ'>, C<'パ'>,
and hankaku katakana with a voiced mark
or a semi-voiced mark, such as C<'ｶﾞ'>, C<'ﾊﾟ'>,
is counted as 1.
Similarly, the conversion between zenkaku hiragana C<'う゛'> and
zenkaku katakana C<'ヴ'> is counted as 1.

=over 4

=item C<kanaH2Z(STRING or SCALAR REF)>

=item C<kataH2Z(STRING or SCALAR REF)>

Converts Hankaku Katakana to Zenkaku Katakana

B<Note:> C<kataH2Z> is an alias of C<kanaH2Z>.

=item C<kataZ2H(STRING or SCALAR REF)>

Converts Zenkaku Katakana to Hankaku Katakana

=item C<kanaZ2H(STRING or SCALAR REF)>

Converts Zenkaku Hiragana and Katakana to Hankaku Katakana

=item C<hiXka(STRING or SCALAR REF)>

Converts Zenkaku Hiragana to Zenkaku Katakana
and Zenkaku Katakana to Zenkaku Hiragana at once.

=item C<hi2ka(STRING or SCALAR REF)>

Converts Zenkaku Hiragana to Zenkaku Katakana

=item C<ka2hi(STRING or SCALAR REF)>

Converts Zenkaku Katakana to Zenkaku Hiragana

=back

=head2 Conversion of Whitespace Characters

If a reference to a scalar variable is specified as the first argument,
the string referred to it is converted and
the number of characters replaced is returned.
Otherwise, returns a string converted and
the specified string is unaffected.

=over 4

=item C<spaceH2Z(STRING or SCALAR REF)>

Converts space (half-width) to ideographic space (full-width)
in the specified string and returns the converted string.

=item C<spaceZ2H(STRING or SCALAR REF)>

Converts ideographic space (full-width) to space (half-width)
in the specified string and returns the converted string.

=back

=head1 CAVEAT

A legal Shift_JIS character in this module
must match the following regular expression:

   [\x00-\x7F\xA1-\xDF]|[\x81-\x9F\xE0-\xFC][\x40-\x7E\x80-\xFC]

Any string from an external source should be checked by C<issjis()>
function, excepting you know it is surely coded in Shift_JIS.

Use of an illegal Shift_JIS string may lead to odd results.

Some Shift_JIS double-byte characters have a trailing byte
in the range of C<[\x40-\x7E]>, viz.,

   @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

The Perl lexer (parhaps) doesn't take any care to these bytes,
so they sometimes make trouble.
e.g. the quoted literal C<"表"> causes a B<fatal error>,
since its trailing byte C<0x5C> backslashes the closing quote.

Such a problem doesn't arise when the string is gotten from
any external resource.
But writing the script containing Shift_JIS
double-byte characters needs the greatest care.

The use of single-quoted heredoc, C<E<lt>E<lt> ''>,
or C<\xhh> meta characters is recommended
in order to define a Shift_JIS string literal.

The safe ASCII-graphic characters, C<[\x21-\x3F]>, are:

   !"#$%&'()*+,-./0123456789:;<=>?

They are preferred as the delimiter of quote-like operators.

=head1 BUGS

This library supposes C<$[> is always equal to 0, never 1.

=head1 AUTHOR

Tomoyuki SADAHIRO

  bqw10602@nifty.com
  http://homepage1.nifty.com/nomenclator/perl/

  Copyright(C) 2001-2002, SADAHIRO Tomoyuki. Japan. All rights reserved.

  This module is free software; you can redistribute it
  and/or modify it under the same terms as Perl itself.

=head1 SEE ALSO

=over 4

=item L<ShiftJIS::Regexp>

=item L<ShiftJIS::Collate>

=item L<String::Multibyte>

=back

=cut
