NAME

    App::migrate - upgrade / downgrade project

VERSION

    This document describes App::migrate version v0.2.3

SYNOPSIS

        use App::migrate;
    
        my $migrate = App::migrate->new()
        $migrate = $migrate->load($file)
    
        @paths   = $migrate->find_paths($v_from => $v_to)
        say "versions: @{$_}" for @paths;
    
        @steps   = $migrate->get_steps($paths[0])
        for (@steps) {
          say "$_->{prev_version} ... $_->{next_version}";
          if ($_->{type} eq 'VERSION' or $_->{type} eq 'RESTORE') {
              say "$_->{type} $_->{version}";
          } else {
              say "$_->{type} $_->{cmd} @{$_->{args}}";
          }
        }
    
        $migrate = $migrate->on(BACKUP  => sub{ my $step=shift; return or die });
        $migrate = $migrate->on(RESTORE => sub{ my $step=shift; return or die });
        $migrate = $migrate->on(VERSION => sub{ my $step=shift; return or die });
        $migrate = $migrate->on(error   => sub{ my $step=shift; return or die });
        $migrate->run($paths[0]);

DESCRIPTION

    If you're looking for command-line tool - see migrate. This module is
    actual implementation of that tool's functionality and you'll need it
    only if you're developing similar tool (like narada-install) to
    implement specifics of your project in single perl script instead of
    using several external scripts.

    This module implements file format (see "SYNTAX") to describe sequence
    of upgrade and downgrade operations needed to migrate something between
    different versions, and API to analyse and run these operations.

    The something mentioned above is usually some project, but it can be
    literally anything - OS configuration in /etc, or overall OS setup
    including installed packages, etc. - anything what has versions and
    need complex operations to upgrade/downgrade between these versions.
    For example, to migrate source code you can use VCS like Git or
    Mercurial, but they didn't support empty directories, file permissions
    (except executable), non-plain file types (FIFO, UNIX socket, etc.),
    xattr, ACL, configuration files which must differ on each site, and
    databases. So, if you need to migrate anything isn't supported by VCS -
    you can try this module/tool.

    Sometimes it isn't possible to really downgrade because some data was
    lost while upgrade - to handle these situations you should provide a
    ways to create complete backup of your project and restore any
    project's version from these backups while downgrade (of course,
    restoring backups will result in losing new changes, so whenever
    possible it's better to do some extra work to provide a way to
    downgrade without losing any data).

 Example

    Here is example how to run migration from version '1.1.8' to '1.2.3' of
    some project which uses even minor versions '1.0.x' and '1.2.x' for
    stable releases and odd minor versions '1.1.x' for unstable releases.
    The nearest common version between '1.1.8' and '1.2.3' is '1.0.42',
    which was the parent for both '1.1.x' and '1.2.x' branches, so we need
    to downgrade project from '1.1.8' to '1.0.42' first, and then upgrade
    from '1.0.42' to '1.2.3'. You'll need two *.migrate files, one which
    describe migrations from '1.0.42' (or earlier version) to '1.1.8', and
    another with migrations from '1.0.42' (or earlier) to '1.2.3'. For
    brevity let's not make any backups while migration.

        my $migrate = App::migrate
            ->new
            ->load('1.1.8.migrate')
            ->load('1.2.3.migrate');
        $migrate
            ->on(BACKUP => sub {})
            ->run( $migrate->find_paths('1.1.8' => '1.2.3') );

INTERFACE

 new

        $migrate = App::migrate->new;

    Create and return new App::migrate object.

 load

        $migrate->load('path/to/migrate');

    Load migration commands into $migrate object.

    You should load at least one file with migration commands before you
    can use "find_paths", "get_steps" or "run".

    When loading multiple files, if they contain two adjoining 'VERSION'
    operations with same version values then migration commands between
    these two version values will be used from first loaded file containing
    these version values.

    Will throw if given file's contents don't conform to "Specification" -
    this may be used to check file's syntax.

 find_paths

        @paths = $migrate->find_paths($from_version => $to_version);

    Find and return all possible paths to migrate between given versions.

    If no paths found - return empty list. This may happens because you
    didn't loaded migrate files which contain required migrations or
    because there is no way to migrate between these versions (for example,
    if one of given versions is incorrect).

    Multiple paths can be found, for example, when your project had some
    branches which was later merged.

    Each found path returned as single ARRAYREF element in returned list.
    This ARRAYREF contains list of all intermediate versions, one by one,
    starting from $from_version and ending with $to_version.

    For example, if our project have this version history:

            1.0.0
              |
            1.0.42
             / \
        1.1.0   1.2.0
          |       |
        1.1.8   1.2.3
          | \     |
          |  \----|
        1.1.9   1.2.4
          |       |
        1.1.10  1.2.5

    then you'll probably have these migrate files:

        1.1.10.migrate          1.0.0->…->1.0.42->1.1.0->…->1.1.10
        1.2.5.migrate           1.0.0->…->1.0.42->1.2.0->…->1.2.3->1.2.4->1.2.5
        1.1.8-1.2.4.migrate     1.0.0->…->1.0.42->1.1.0->…->1.1.8->1.2.4

    If you "load" files 1.2.5.migrate and 1.1.8-1.2.4.migrate and then call
    find_paths('1.0.42' => '1.2.5'), then it will return this list with two
    paths (in any order):

        (
            ['1.0.42', '1.1.0', …, '1.1.8', '1.2.4', '1.2.5'],
            ['1.0.42', '1.2.0', …, '1.2.3', '1.2.4', '1.2.5'],
        )

 get_steps

        @steps = $migrate->get_steps( \@versions );

    Return list of all migration operations needed to migrate on path given
    in @versions.

    For example, to get steps for first path returned by "find_paths":

        @steps = $migrate->get_steps( $migrate->find_paths($from=>$to) );

    Steps returned in order they'll be executed while "run" for this path.
    Each element in @steps is a HASHREF with these keys:

        type    => one of these values:
                    'VERSION', 'before_upgrade', 'upgrade',
                    'downgrade', 'after_downgrade', 'RESTORE'
    
        # these keys exists only if value of type key is one of:
        #   VERSION, RESTORE
        version => version number
    
        # these keys exists only if value of type key is one of:
        #   before_upgrade, upgrade, downgrade, after_downgrade
        cmd     => command to run
        args    => ARRAYREF of params for that command

    Will throw if unable to return requested steps.

 on

        $migrate = $migrate->on(BACKUP  => \&your_handler);
        $migrate = $migrate->on(RESTORE => \&your_handler);
        $migrate = $migrate->on(VERSION => \&your_handler);
        $migrate = $migrate->on(error   => \&your_handler);

    Set handler for given event.

    All handlers will be called only by "run"; they will get single
    parameter - step HASHREF (BACKUP handler will get step in same format
    as RESTORE), see "get_steps" for details of that HASHREF contents. Also
    these handlers may use $ENV{MIGRATE_PREV_VERSION} and
    $ENV{MIGRATE_NEXT_VERSION} - see "run" for more details.

    'BACKUP' event

      Handler will be executed when project backup should be created:
      before starting any new migration, except next one after RESTORE.

      If handler throws then 'error' handler will be executed.

      Default handler will throw (because it doesn't know how to backup
      your project).

      NOTE: If you'll use handler which doesn't really create and keep
      backups for all versions then it will be impossible to do RESTORE
      operation.

    'RESTORE' event

      Handler will be executed when project should be restored from backup:
      when downgrading between versions which contain RESTORE operation or
      when migration fails.

      If handler throws then 'error' handler will be executed.

      Default handler will throw (because it doesn't know how to restore
      your project).

    'VERSION' event

      Handler will be executed after each successful migration.

      If handler throws then 'error' handler will be executed.

      Default handler does nothing.

    'error' event

      Handler will be executed when one of commands executed while
      migration fails or when BACKUP, RESTORE or VERSION handlers throw.

      If handler throws then try to restore version-before-migration
      (without calling error handler again if it throws too).

      Default handler will run $SHELL (to let you manually fix errors) and
      throw if you $SHELL exit status != 0 (to let you choose what to do
      next - continue migration if you fixed error or interrupt migration
      to restore version-before-migration from backup).

 run

        $migrate->run( \@versions );

    Will use "get_steps" to get steps for path given in @versions and
    execute them in order. Will also call handlers as described in "on".

    Before executing each step will set $ENV{MIGRATE_PREV_VERSION} to
    current version (which it will migrate from) and
    $ENV{MIGRATE_NEXT_VERSION} to version it is trying to migrate to.

SYNTAX

 Goals

    Syntax of this file was designed to accomplish several goals:

      * Be able to automatically make sure each 'upgrade' operation has
      corresponding 'downgrade' operation (so it won't be forget - but, of
      course, it's impossible to automatically check is 'downgrade'
      operation will correctly undo effect of 'upgrade' operation).

      Thus custom file format is needed.

      * Make it easier to manually analyse is 'downgrade' operation looks
      correct for corresponding 'upgrade' operation.

      Thus related 'upgrade' and 'downgrade' operations must go one right
      after another.

      * Make it obvious some version can't be downgraded and have to be
      restored from backup.

      Thus RESTORE operation is named in upper case.

      * Given all these requirements try to make it simple and obvious to
      define migrate operations, without needs to write downgrade code for
      typical cases.

      Thus it's possible to define macro to turn combination of
      upgrade/downgrade operations into one user-defined operation (no
      worries here: these macro doesn't support recursion, it isn't
      possible to redefine them, and they have lexical scope - from
      definition to the end of this file - so they won't really add
      complexity).

 Example

        VERSION 0.0.0
        # To upgrade from 0.0.0 to 0.1.0 we need to create new empty file and
        # empty directory.
        upgrade     touch   empty_file
        downgrade   rm      empty_file
        upgrade     mkdir   empty_dir
        downgrade   rmdir   empty_dir
        VERSION 0.1.0
        # To upgrade from 0.1.0 to 0.2.0 we need to drop old database. This
        # change can't be undone, so only way to downgrade from 0.2.0 is to
        # restore 0.1.0 from backup.
        upgrade     rm      useless.db
        RESTORE
        VERSION 0.2.0
        # To upgrade from 0.2.0 to 1.0.0 we need to run several commands,
        # and after downgrading we need to kill some background service.
        before_upgrade
          patch    <0.2.0.patch >/dev/null
          chmod +x some_daemon
        downgrade
          patch -R <0.2.0.patch >/dev/null
        upgrade
          ./some_daemon &
        after_downgrade
          killall -9 some_daemon
        VERSION 1.0.0
    
        # Let's define some lazy helpers:
        DEFINE2 only_upgrade
        upgrade
        downgrade true
    
        DEFINE2 mkdir
        upgrade
          mkdir "$@"
        downgrade
          rm -rf "$@"
    
        # ... and use it:
        only_upgrade
          echo "Just upgraded to $MIGRATE_NEXT_VERSION"
    
        VERSION 1.0.1
    
        # another lazy macro (must be defined above in same file)
        mkdir dir1 dir2
    
        VERSION 1.1.0

 Specification

    Recommended name for file with upgrade/downgrade operations is either
    migrate or <version>.migrate.

    Each line in migrate file must be one of these:

      * line start with symbol "#"

      For comments. Line is ignored.

      * line start with any non-space symbol, except "#"

      Contain one or more elements separated by one or more space symbols:
      operation name (case-sensitive), zero or more params (any param may
      be quoted, params which contain one of 5 symbols "\\\"\t\r\n" must be
      quoted).

      Quoted params must be surrounded by double-quote symbol, and any of
      mentioned above 5 symbols must be escaped like shown above.

      * line start with two spaces

      Zero or more such lines after line with operation name form one more,
      multiline, extra param for that operation (first two spaces will be
      removed from start of each line before providing this param to
      operation).

      Not all operations may have such multiline param.

      * empty line

      If this line is between operations then it's ignored.

      If this line is inside operation's multiline param - then that
      multiline param will include this empty line.

      If you will need to include empty line at end of multiline param then
      you'll have to use line with two spaces instead.

    While executing any commands two environment variables will be set:
    $MIGRATE_PREV_VERSION and $MIGRATE_NEXT_VERSION (first is always
    version we're migrating from, and second is always version we're
    migrating to - i.e. while downgrading $MIGRATE_NEXT_VERSION will be
    lower/older version than $MIGRATE_PREV_VERSION)

    All executed commands must complete without error, otherwise emergency
    shell will be started and user should either fix the error and exit
    from shell to continue migration, or exit 1 from shell to interrupt
    migration and restore previous-before-this-migration version from
    backup.

 Supported operations

  VERSION

    Must have exactly one param (version number). Some symbols are not
    allowed in version numbers: special (0x00-0x1F,0x7F), both slash, all
    three quotes, ?, * and space.

    Multiline param not supported.

    This is delimiter between sequences of migrate operations.

    Each file must contain 'VERSION' operation before any migrate
    operations (i.e. before first 'VERSION' operation only 'DEFINE',
    'DEFINE2' and 'DEFINE4' operations are allowed).

    All operations after last 'VERSION' operation will be ignored.

  before_upgrade

  upgrade

  downgrade

  after_downgrade

    These operations must be always used in pairs: first must be one of
    'before_upgrade' or 'upgrade' operation, second must be one of
    'downgrade' or 'after_downgrade' or 'RESTORE' operations.

    These four operations may have zero or more params and optional
    multiline param. If they won't have any params at all they'll be
    processed like they have one (empty) multiline param.

    Their params will be executed as a single shell command at different
    stages of migration process and in different order:

      * On each migration only commands between two nearest VERSION
      operations will be processed.

      * On upgrading (migrate forward from previous VERSION to next
      VERSION) will be executed all 'before_upgrade' operations in forward
      order then all 'upgrade' operations in forward order.

      * On downgrading (migrate backward from next VERSION to previous)
      will be executed all 'downgrade' operations in backward order, then
      all 'after_downgrade' operations in backward order.

    Shell command to use will be:

      * If operation has one or more params - first param will become
      executed command name, other params will become command params.

      If operation also has multiline param then it content will be saved
      into temporary file and name of that file will be added at end of
      command's params.

      * Else multiline param will be saved into temporary file (after
      shebang #!/path/to/bash -ex if first line of multiline param doesn't
      start with #!), which will be made executable and run without any
      params.

  RESTORE

    Doesn't support any params, neither usual nor multiline.

    Can be used only after 'before_upgrade' or 'upgrade' operations.

    When one or more 'RESTORE' operations are used between some 'VERSION'
    operations then all 'downgrade' and 'after_downgrade' operations
    between same 'VERSION' operations will be ignored and on downgrading
    previous version will be restored from backup.

  DEFINE

    This operation must have only one non-multiline param - name of defined
    macro. This name must not be same as one of existing operation names,
    both documented here or created by one of previous 'DEFINE' or
    'DEFINE2' or 'DEFINE4' operations.

    Next operation must be one of 'before_upgrade', 'upgrade', 'downgrade'
    or 'after_downgrade' - it will be substituted in place of all next
    operations matching name of this macro.

    When substituting macro it may happens what both this macro definition
    have some normal params and multiline param, and substituted operation
    also have some it's own normal params and multiline param. All these
    params will be combined into single command and it params in this way:

      * If macro definition doesn't have any params - params of substituted
      operation will be handled as usually for 'upgrade' etc. operations.

      * If macro definition have some params - they will be handled as
      usually for 'upgrade' etc. operations, so we'll always get some
      command and optional params for it.

      Next, all normal params of substituted command (if any) will be
      appended to that command params.

      Next, if substituted command have multiline param then it will be
      saved to temporary file and name of that file will be appended to
      that command params.

  DEFINE2

    Work similar to DEFINE, but require two next operations after it: first
    must be one of 'before_upgrade' or 'upgrade', and second must be one of
    'downgrade' or 'after_downgrade'.

    Params of both operations will be combined with params of substituted
    operation as explained above.

  DEFINE4

    Work similar to DEFINE, but require four next operations after it:
    first must be 'before_upgrade', second - 'upgrade', third -
    'downgrade', fourth - 'after_downgrade'.

    Params of all four operations will be combined with params of
    substituted operation as explained above.

SUPPORT

 Bugs / Feature Requests

    Please report any bugs or feature requests through the issue tracker at
    https://github.com/powerman/migrate/issues. You will be notified
    automatically of any progress on your issue.

 Source Code

    This is open source software. The code repository is available for
    public review and contribution under the terms of the license. Feel
    free to fork the repository and submit pull requests.

    https://github.com/powerman/migrate

        git clone https://github.com/powerman/migrate.git

 Resources

      * MetaCPAN Search

      https://metacpan.org/search?q=App-migrate

      * CPAN Ratings

      http://cpanratings.perl.org/dist/App-migrate

      * AnnoCPAN: Annotated CPAN documentation

      http://annocpan.org/dist/App-migrate

      * CPAN Testers Matrix

      http://matrix.cpantesters.org/?dist=App-migrate

      * CPANTS: A CPAN Testing Service (Kwalitee)

      http://cpants.cpanauthors.org/dist/App-migrate

AUTHOR

    Alex Efros <powerman@cpan.org>

COPYRIGHT AND LICENSE

    This software is Copyright (c) 2015- by Alex Efros <powerman@cpan.org>.

    This is free software, licensed under:

      The MIT (X11) License

