# Copyright (C) 2001-2007, The Perl Foundation.
# $Id: pdd04_datatypes.pod 18563 2007-05-16 00:53:55Z chromatic $

=head1 NAME

docs/pdds/pdd04_datatypes.pod - Parrot's internal data types

=head1 ABSTRACT

This PDD describes Parrot's internal data types.

{{ NOTE: this is a good overview, but we need more complete
specifications of the behavior of the datatypes. }}

=head1 DESCRIPTION

This PDD details the basic datatypes that the Parrot core knows how to deal 
with. Three of these (the integer, floating point and string datatypes)  have
no additional semantics. The fourth datatype, the Parrot Magic Cookie (PMC)
acts as the basis for all high level languages running on top of  Parrot; only
the most basic aspects are described here.

Note that PMC and string internals are volatile and may be changed in the
future (although this will become increasingly unlikely as we near v1.0).
Access from external code to the internals of particular datatypes should be
via the extension mechanism (see F<docs/pdds/pdd11_extending.pod>,  which has
more explicit guarantees of stability.

=head1 IMPLEMENTATION

=head2 Integer data types

Integer data types are generically referred to as C<INT>s.  These are whatever
size native integer was chosen at Parrot configuration time. The C-level
typedefs C<INTVAL> and C<UINTVAL> get you a platform-native signed and unsigned
integer respectively.

=head2 Floating point data types

Floating point data types are generically referred to as C<NUM>s.  These are
whatever size float was chosen when parrot was configured. The C level typedef
C<FLOATVAL> will get you one of these.

=head2 String data types

Parrot has a single internal string form:

    struct parrot_string_t {
        pobj_t obj;
        UINTVAL bufused;
        void *strstart;
        UINTVAL strlen;
        const ENCODING *encoding;
        const CHARTYPE *type;
        INTVAL language;
    }

The fields are:

=over 4

=item obj

A pointer to a Parrot object, Parrot's most general internal data type. In this
case, it holds the buffer for the string data, the size of the buffer in bytes,
and any applicable flags.

=item bufused

The amount of the buffer currently in use, in bytes.

=item strstart

A pointer to the beginning of the actual string (which may not be positioned 
at the start of the buffer).

=item strlen

The length of the string, in characters.

=item encoding

How the data is encoded (e.g. fixed 8-bit characters, UTF-8, or UTF-32).  Note
that this specifies encoding only -- it's valid to encode  EBCDIC characters
with the UTF-8 algorithm. Silly, but valid.

The ENCODING structure specifies the encoding (by index number and by name, for
ease of lookup), the maximum number of bytes that a single character will
occupy in that encoding, as well as functions for manipulating strings with
that encoding.

=item type

What sort of string data is in the buffer, for example ASCII, EBCDIC, or
Unicode.

The CHARTYPE structure specifies the character type (by index number and by 
name) and provides functions for transcoding to and from that character type.

=item language

This specifies the language corresponding to the string. This is to allow for
locale-based data to be attached to strings. To give an example of the use of
this: strings in German may not sort in the same way as strings in French, even
when both types use the Latin-1 charset and are encoded in UTF-8.

Note that language-agnostic utilities are at liberty to ignore this entry.

=back

=head2 Parrot Magic Cookies (PMCs)

Parrot Magic Cookies, or PMCs, are the last of Parrot's basic datatypes, but
are also potentially the most important. Their basic structure is as follows.
All PMCs have the form:

    struct PMC {
        pobj_t obj;
        VTABLE *vtable;
 #if ! PMC_DATA_IN_EXT
        DPOINTER *data;
 #endif
        struct PMC_EXT *pmc_ext;
    };

where C<obj> is a pointer to an C<pobj_t> structure:

    typedef struct pobj_t {
        UnionVal u;
        Parrot_UInt flags;
 #if ! DISABLE_GC_DEBUG
        UINTVAL _pobj_version;
 #endif
    } pobj_t;

and where:

    typedef union UnionVal {
        struct {
            void * _bufstart;
            size_t _buflen;
        } _b;
        struct {
            DPOINTER* _struct_val;
            PMC* _pmc_val;
        } _ptrs;
        INTVAL _int_val;
        FLOATVAL _num_val;
        struct parrot_string_t * _string_val;
    } UnionVal;

C<u> holds data associated with the PMC. This can be in the form of an  integer
value, a floating point value, a string value, or a pointer  to other data.
C<u> may be empty, since the PMC structure also provides a more general data
pointer, but is useful for PMCs which hold only a single piece of data (e.g.
C<PerlInts>).

C<flags> holds a set of flags associated with the PMC; these are documented  in
F<include/parrot/pobj.h>, and are generally only used within the Parrot 
internals.

C<_pobj_version> is only used for debugging Parrot's garbage collector. It is
documented elsewhere (well, it will be once we get around to doing that...).

C<vtable> holds a pointer to the B<vtable> associated with the PMC. This points
to a set of functions, with interfaces described in
F<docs/pdds/pdd02_vtables.pod> that implement the basic behaviour of the PMC
(i.e. how it behaves under addition, subtraction, cloning etc.)

C<data> (if present) holds a pointer to any additional data associated  with
the PMC. This may be NULL.

C<pmc_ext> points to an extended PMC structure. This has the form:

    struct PMC_EXT {
 #if PMC_DATA_IN_EXT
        DPOINTER *data;
 #endif
        PMC *_metadata;
        struct _Sync *_synchronize;
        PMC *_next_for_GC;
    };

C<data> is a generic data pointer, as described above.

C<_metadata> holds internal PMC metadata. The specification for this has not
yet been finalized.

C<_synchronize> is for access synchronization between shared PMCs.

C<_next_for_GC> determines the next PMC in the 'used' list during dead object 
detection in the GC.

PMCs are not required to have a C<PMC_EXT> structure (i.e. C<pmc_ext> can be
null).

PMCs are used to implement the basic data types of the high level languages
running on top of Parrot. For instance, a Perl 5 C<SV> will map onto one (or
more) types of PMC, while particular Python datatypes will map onto different
types of PMC.

=head2 Vtable Overloading

PMCs may declare vtable methods. The following list details the raw method names:

=over 4

=item init

Called when an object is first created.

=item init_pmc

Alternative entry point called when an object is first created.  Accepts a PMC
parameter used to initialize the given object.  Interpretation of the PMC is
PMC-specific.

NOTE: It is strongly suggested that init_pmc(PMCNULL) be equivalent to
init(), though there will of necessity be exceptions.

=item morph

=item mark

Called when the DOD is tracing live PMCs. If this method is called then the
code must mark all strings and PMCs that it contains as live, otherwise they
may be collected.

This method is only called if the PMC is flagged as having a special mark
routine, and is not necessary for normal objects.

=item destroy

Called when the PMC is destroyed. This method is only called if the PMC is
marked as having an active finalizer.

=item clone

Clone a PMC.

=item getprop

=item setprop

=item delprop

=item getprops

=item type

=item type_keyed

=item type_keyed_int

=item type_keyed_str

=item subtype

=item name

=item find_method

=item get_integer

Return the integer value of the object

=item get_integer_keyed

=item get_integer_keyed_int

=item get_integer_keyed_str

=item get_number

Return the floating-point value of the object

=item get_number_keyed

=item get_number_keyed_int

=item get_number_keyed_str

=item get_bignum

Return the extended precision numeric value of the PMC

=item get_string

Return the string value of the PMC

=item get_string_keyed

=item get_string_keyed_int

=item get_string_keyed_str

=item get_bool

Return the true/false value of the PMC

=item get_bool_keyed

=item get_bool_keyed_int

=item get_bool_keyed_str

=item get_pmc

Return the PMC for this PMC.

=item get_pmc_keyed

=item get_pmc_keyed_int

=item get_pmc_keyed_str

=item get_pointer

=item get_pointer_keyed

=item get_pointer_keyed_int

=item get_pointer_keyed_str

=item set_integer_native

Set the integer value of this PMC

=item set_integer_same

=item set_integer_keyed

=item set_integer_keyed_int

=item set_integer_keyed_str

=item set_number_native

Set the floating-point value of this PMC

=item set_number_same

=item set_number_keyed

=item set_number_keyed_int

=item set_number_keyed_str

=item set_bignum_int

Set the extended-precision value of this PMC

=item set_string_native

Set the string value of this PMC

=item set_string_same

=item set_string_keyed

=item set_string_keyed_int

=item set_string_keyed_str

=item set_bool

Set the true/false value of this PMC

=item assign_pmc

Set the value to the value of the passed in

=item set_pmc

Make the PMC refer to the PMC passed in

=item set_pmc_keyed

=item set_pmc_keyed_int

=item set_pmc_keyed_str

=item set_pointer

=item set_pointer_keyed

=item set_pointer_keyed_int

=item set_pointer_keyed_str

=item elements

Return the number of elements in the PMC, if the PMC is treated as an
aggregate.

=item pop_integer

=item pop_float

=item pop_string

=item pop_pmc

=item push_integer

=item push_float

=item push_string

=item push_pmc

=item shift_integer

=item shift_float

=item shift_string

=item shift_pmc

=item unshift_integer

=item unshift_float

=item unshift_string

=item unshift_pmc

=item splice

=item add

=item add_int

=item add_float

=item subtract

=item subtract_int

=item subtract_float

=item multiply

=item multiply_int

=item multiply_float

=item divide

=item divide_int

=item divide_float

=item modulus

=item modulus_int

=item modulus_float

=item cmodulus

=item cmodulus_int

=item cmodulus_float

=item neg

=item bitwise_or

=item bitwise_or_int

=item bitwise_and

=item bitwise_and_int

=item bitwise_xor

=item bitwise_xor_int

=item bitwise_ors

=item bitwise_ors_str

=item bitwise_ands

=item bitwise_ands_str

=item bitwise_xors

=item bitwise_xors_str

=item bitwise_not

=item bitwise_shl

=item bitwise_shl_int

=item bitwise_shr

=item bitwise_shr_int

=item concatenate

=item concatenate_native

=item is_equal

=item is_same

=item cmp

=item cmp_num

=item cmp_string

=item logical_or

=item logical_and

=item logical_xor

=item logical_not

=item repeat

=item repeat_int

=item increment

=item decrement

=item exists_keyed

=item exists_keyed_int

=item exists_keyed_str

=item defined

=item defined_keyed

=item defined_keyed_int

=item defined_keyed_str

=item dtem delete_keyed_str

=item nextkey_keyed

=item nextkey_keyed_itr_str

=item invoke

=item can

=item does

=item isa

=item fsh

=item visit

=item share

=item add_method

=item add_attribute

=item add_parent

=item add_role

=back


=head2 Interaction between PMCs and high-level objects

{{ Address the problem of high-level objects inheriting from low-level PMCs,
and any structural changes to low-level PMCs that might require. }}


=head1 ATTACHMENTS

None.

=head1 REFERENCES

The perl modules Math::BigInt and Math::BigFloat. Alex Gough's suggestions for
bigint/bignum implementation, outlined in F<docs/pdds/pdd14_bignum.pod>. The
Unicode standard at L<http://www.unicode.org>.

=head1 GLOSSARY

=over 4

=item Type

Type refers to a basic Parrot data type. There are four such: integers,
floating point numbers (often just numbers), strings and Parrot Magic Cookies
(PMCs).

=back

=head1 VERSION

1.4

=head2 CURRENT

     Maintainer: Dan Sugalski <dan@sidhe.org>
     Class: Internals
     PDD Number: 4
     Version: 1.5
     Status: Developing
     Last Modified: 11 June 2005
     PDD Format: 1
     Language: English

=head2 HISTORY

=over 4

=item Version 1.5, 11 June 2005

=item Version 1.4, 20 February 2004

=item Version 1.3, 2 July 2001

=item Version 1.2, 2 July 2001

=item Version 1.1, 2 March 2001

=item Version 1, 1 March 2001

=back

=head1 CHANGES

=over 4

=item Version 1.5

Removed BigInt and BigNum from the definition of I* and N* registers --
according to Leo they are now always PMCs, never register types of their own.

=item Version 1.4

Document basic PMC internals. Make clear the fact that the bigint/bignum
description is still provisional. Other minor fixups to make the  documentation
match reality.

=item Version 1.3

Fixed some silly typos and dropped phrases.

Took all the underscores out of the field names.

=item Version 1.2

The string header format has changed some to allow for type tagging. The flags
information for strings has changed as well.

=item Version 1.1

INT and NUM are now concepts rather than data structures, as making them data
structures was a Bad Idea.

=item Version 1

None. First version

=back

