=head1 NAME 

TODO LIST FOR WORDNET::SIMILARITY

=head1 SYNOPSIS

A list of things to do for WordNet Similarity. 

=head1 DESCRIPTION

As these items are completed, move them down into Recently
Completed Items, make sure to date and initial. When we have
a version release, all of the recently completed items 
should be moved into changelog.pod. 

=head2 FOR VERSION 0.09:

=over

=item *

The trace output for certain measures appears to "stutter".  For example,
for the path measure, comparing beer#n#1 and wine#n#1 will result in
the trace containing the same shortest path several times.

=item *

Repair addConfigOption(). addConfigOption is a class function in Similarity.
This causes problems when two different measures define an option of the
same name and then a script tries to C<use> both of those measure modules.
For example, both vector and lesk have an option called 'relation'.  When
a script, like access.t, tries to use both measures, an error occurs
because one script is attempting to redefine the option of another.

=item * 

Standardize "path to wordnet" handling. utilities should have an option
to specify the path, and then if it is not go through the same looking
procedure. wnDepths.pl and *Freq.pl, for example, do this somewhat
differently. Let's standardize, maybe even putting into a module. 

=item *

re-write *Freq.pl programs to reduce redundancy and make faster. At 
present there are bugs in all of these programs. Create test cases that 
are manually verified and included in testing directory. 

=item *

The leather_carp mystery: leather_carp occurs 3 times in the BNC. It has
only one sense, and it is a leaf node. By all rights both standard and
resnik counting should agree. They do not, standard counting seems to
overcount by a factor of 2. 

=item *

6412325 is the synset for i in wordnet.

 ic-brown-add1.dat:6412325n 10497
 ic-brown.dat:6412325n 10496
 ic-brown-resnik-add1.dat:6412325n 1750.33333333357

there are three senses of 'i' in wordnet. Why isn't the count in the last
line 3498.6. The count shown suggests that i has 6 senses, which it 
doesn't...[TDP 12/28/03 - the overcounting shown in the leather_carp
problem suggests that the resnik count above *might* be correct. 
1750.3 * 3 = 5251 - if that is the number of occurrences of i in the
brown corpus, then the resnik count is fine. The overcounting still
exists in the standard method however.

=item * 

run test case for rawtextFreq.pl. Use a single word in a file (like I), 
and make sure that default and resnik counting work as expected. Ted ran a 
test case where the only word in the file was "I" (with no stop list)
and the counts were all 2 (both at leaf and up to ROOT). The same occurred 
with brownFreq.pl and will presumably affect all the Freq.pl programs.

=item * 

improve error handling in ***Freq.pl programs, such that  if they do not  
get the input they require they issue an error  message and  do not run  
endlessly. 

=item * 

support --trace option on info content programs to allow for wps format to 
be displayed in addition to (or instead of?) offset. 

=item * 

run profiles of rawtextFreq.pl and BNCFreq.pl to determine where time is 
being spent. Brown, SemCor and Treebank all seem to run reasonably 
quickly (20 minutes, 5 minutes, and 40 minutes, respectively). Run 1 
million words worth of BNC in order to compare with Brown and Treebank.

=item * 

rawtextFreq.pl runs really slowly. It may have to do with the fact that 
raw text has no markup in the text to identify sentence boundaries or 
otherwise guide the programs. This might particularly slow down compound 
identification. 

=item * 

Makefile.PL and semCorFreq.pl seem to be somewhat alike. Can Makefile.PL
simply call /utils/SemCorFreq.pl, or can this duplication be avoided in
some other way?

=item *

update documentation to warn users that Freq.pl programs convert text to
lower case (verify that this is true for rawtextFreq.pl) and that 
stoplists must also be all lowercase. 

=back

=head2 FOR VERSION 0.10:

=over 4

=item * 

speed up lesk, and make it more generic. string matching is the  big offender with respect to speed, and wordnet specific  stuff is the  
problem with respect to generality. 

=item * 

update lesk/vector to support new relations in WordNet 2.0

=back

=head2 FOR VERSION 0.11:

=over 4

=item * 

update hso to support new relations in WordNet 2.0

=item *

update hso to support the use of a hypothetical root node. Currently (as 
of version 0.06 and 0.07) its paths (for hypernyms) are limited to a 
particular taxonomy. This might be problematic when it comes to nouns, 
which are split into 9(?) separate taxonomies within wordnet. And of
course verbs are split into hundreds of taxonomies. Right now when hso
is on a hypernym path it isn't able to cross "up and over". Seems like it 
should be able to do so. 

=back

=head2 FOR VERSION 1.00:

=over 4

=item * 

re-write hso to make it faster and more generic. check to see if hso uses  
hypo root node, and consider adding ability to turn on/off. 

=back

=head2 GOOD IDEAS FOR FUTURE WORK, DO WHEN POSSIBLE

=over 4

=item *

edge/path and jcn are both distance measures. To convert them to 
similarity measures, we currently use 1/distance. This shifts the scale of 
the measures and changes the relative distance between pairs. Alternatives
are to use -dist or maxdist-dist. Computation of maxdist for path is much 
like computation for lch (with and without hypo root node). for jcn it 
poses a new issue, in that we would need to find the pair of concepts that 
had the greatest individual information content, and are subsumed by a 
root node (either hypo or "real").

=item * 

store WN version in vector DB file and warn when version is different

=item * 

check if warnings are issued when there are version clashes  between info  
content files and wordnet version. 

=item *

Is it possible to have a default value for vectordb? This is the Berkeley
DB file used by vector.pm. It is created from the WordNet glosses, and
is required for vector to run. If vectordb is not specified, or if the
option is specified without a value, vector will fail. (Sid is pursuing 
this.)

=back

=head2 RECENTLY COMPLETED ITEMS

=over

=item *

04/28/2004

=over

=item 1

Created a CGI-based web interface for the relatedness modules. (JM)

=back

=item *

04/19/2004

=over

=item 1

Fixed problem with path to Perl interpreter in Makefile.PL.  This was causing
problems during installation if there was no /usr/local/bin/perl. (JM)

=item 2

wnDepths.pl had forgotten that on Windows some filenames are different;
for example, data.noun is noun.dat. (JM)

=back

=back

=head1 AUTHORS

Copyright (C) 2003-2004 Siddharth Patwardhan, Ted Pedersen, and Jason
Michelizzi. 

=head1 BUGS

None.

=head1 SEE ALSO

changelog.pod

=head1 COPYRIGHT

Copyright (C) 2003-2004 Siddharth Patwardhan, Ted Pedersen, and Jason
Michelizzi. 

Permission is granted to copy, distribute and/or modify this  document  
under the terms of the GNU Free Documentation License, Version 1.2 or any  
later version published by the Free Software Foundation; with no  
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.

Note: a copy of the GNU Free Documentation License is available on the web   
at L<http://www.gnu.org/copyleft/fdl.html> and is included in this    
distribution as FDL.txt. 

=cut

