Authoring Systems Evaluation Criteria


Sun, 10 DEC 95 12:34:18 -0500



At the suggestion of Jools Arnold and with the consent of Nate
Torkington I'll be combining the "Authoring Systems" FAQ with my
expanded "Which is Better" FAQ (expanded beyond TADS and INFORM, that
is).

This provides a wonderful excuse to be even later with the FAQ than
originally estimated, although very early 1995 is still the plan.

I've put together some authoring system rating criteria and invite as
much feedback as you care to provide. The feedback will be
incorporated and acknowledged in the FAQ. Thanks in advance!

*Evaluation Criteria

Based on a subjective analysis of the traffic on rec.arts.int-fiction,
I've developed a set of suggested criteria for evaluating text game
programming systems. I've weighted each criteria for importance on a
1 to 5 scale. In an exercise as approximate as this, I see no point
in expanding the scale to allow for finer gradations of meaning.

Rating categories and weightings are excellent flame-bait. I'd
instead prefer to receive constructive comments so the rating system
can be improved and made more meaningful over time.

The weightings have the following approximate meanings:

5 Critical. The usefulness of the system will be severely compromised
or destroyed if this category does not measure up.
4 Very important. The system does not stand or fall based on this
factor, but usefulness will be restricted if it does not measure up.
3 Important. A system rating poorly in this area will still be useful
but its quality and ease of use will be restricted.
2 Desirable. This area is one that should measure up but is not critical to
usefulness or usability.
1 Nice. An area that is nice-to-have as a convenience for added
utility or ease of use.

*Categories and Weightings

Object Orientation, Weight 3. How closely does the system follow an
object-oriented paradigm? Object-orientation simplifies programming
in a number of important ways.

Ease of Use, Weight 5. This is a catch-all category which takes into
account the "feel" of the system to the author and the
"comfortableness" of use of the system. "Uncomfortable" systems can
be virtually useless.

Source Code, Weight 1. Is source code for the system (as opposed to
the libraries) available? As a practical matter this is only of
importance to dedicated hackers or port maintainers, but can also be
an indicator of the future viability of the system.

Support, Weight 3. How well is the system supported by the author of
the system? How much newsgroup support can you expect?

Literature Library, Weight 3. Most programmers learn by example. If
a large library of source code is available, the system will be easier
to learn and use.

Parser, Weight 5. As the primary interface to the player of the game,
the quality of the parser is critical.

Language Depth, Weight 4. How extensive is the language in terms of
functions, operators, constructs, etc.? How good and useful are the
default libraries? I rate this 4 rather than 5 as it can be a mixed
blessing. A language must have sufficient depth to be useful, and
enough additional depth to be easy to use. However, great depth can
come at the cost of complexity, ultimately reducing ease of use.

Distribution Policy, Weight 3. What sort of restrictions does the
author of the language impose on distribution of games produced with
this system? This category would rate higher if it were not for the
fact that most distribution policies only concern outright commercial
distribution.

Portability, Weight 4. Where will the final game run? On how many
computer systems, of what size and expense? This is a perennial topic
of debate on the newsgroup.

Compiler Platforms, Weight 2. The number of different computer
systems on which the compiler will run, in contrast to the game
itself, receives a lot less attention but nevertheless is of interest
to those prospective authors with lower-end computers.

Compiler Speed, Weight 1. A fast game compiler is nice, of course,
but most authors say that it is not a major factor.

Run Speed, Weight 4. If the game runs slowly and ineffectively, the
game playing audience, usually a fickle and argumentative lot, will
desert ship quickly. Witness the discussion of "Legend" in mid-1995.

Documentation, Weight 5. A poorly documented system is nearly
useless, whereas a system has often been chosen over another merely
due to the quality of the documentation.

Cost, Weight 2. Except in extreme cases, shareware costs have
surprisingly not been very much of an issue.

Game Size, Weight 3. How large a game will the system produce? This
factor might have rated higher except for the fact that most systems
can produce a reasonable size game- of the kind that most authors have
in fact produced. Very large games, requiring years of effort, appear
much less often.

Dynamic Object Creation, Weight 1. Perhaps this does not need a
category of its own, but it has been often discussed as a convenience
factor. A system will be a little more desirable if there is a means
to create and destroy objects during the course of the game, rather
than only at compile time.

bnewell@delphi.com
"Uncle Bob" Newell
Bismarck, North Dakota