zsh - Expansion

Go to the first, previous, next, last section, table of contents.

# Expansion

The types of expansions performed are history expansion, alias expansion, process substitution, parameter expansion, command substitution, arithmetic expansion, brace expansion, filename expansion, and filename generation.

Expansion is done in the above specified order in five steps. The first is history expansion which is only performed in interactive shells. The next step is alias expansion which is done right before the command line is parsed. They are followed by process substitution, parameter expansion, command substitution, arithmetic expansion, and brace expansion which are performed in one step in left-to-right fashion. After these expansions, all unquoted occurrences of the characters `\`, `'`, and `"` are removed and the result is subjected to filename expansion followed by filename generation.

If the `SH_FILE_EXPANSION` option is set, the order of expansion is modified for compatibility with `sh` and `ksh`. Filename expansion is performed immediately after alias substitution, preceding the set of five substitutions mentioned above.

## Filename Expansion

Each word is checked to see if it begins with an unquoted `~`. If it does, then the word up to a `/`, or the end of the word if there is no `/`, is checked to see if it can be substituted in one of the ways described here. If so, then the `~` and the checked portion are replaced with the appropriate substitute value.

A `~` by itself is replaced by the value of the `HOME` parameter. A `~` followed by a `+` or a `-` is replaced by the value of `PWD` or `OLDPWD`, respectively.

A `~` followed by a number is replaced by the directory at that position in the directory stack. `~0` is equivalent to `~+`, and `~1` is the top of the stack. `~+` followed by a number is replaced by the directory at that position in the directory stack. `~+0` is equivalent to `~+`, and `~+1` is the top of the stack. `~-` followed by a number is replaced by the directory that many positions from the bottom of the stack. `~-0` is the bottom of the stack. The `PUSHD_MINUS` option exchanges the effects of `~+` and `~-` where they are followed by a number.

A `~` followed by anything not already covered is looked up as a named directory, and replaced by the value of that named directory if found. Named directories are typically home directories for users on the system. They may also be defined if the text after the `~` is the name of a string shell parameter whose value begins with a `/`. It is also possible to define directory names using the ``-d'` option to the `hash` builtin.

In certain circumstances (in prompts, for instance), when the shell prints a path, the path is checked to see if it has a named directory as its prefix. If so, then the prefix portion is replaced with a `~` followed by the name of the directory. The shortest way of referring to the directory is used, with ties broken in favour of using a named directory, except when the directory is `/`.

If a word begins with an unquoted `=` and the `EQUALS` option is set, the remainder of the word is taken as the name of a command or alias. If a command exists by that name, the word is replaced by the full pathname of the command. If an alias exists by that name, the word is replaced with the text of the alias.

Filename expansion is performed on the right hand side of a parameter assignment, including those appearing after commands of the `typeset` family. In this case, the right hand side will be treated as a colon-separated list in the manner of `PATH` so that a `~` or an `=` following a `:` is eligible for expansion. All such behavior can be disabled by quoting the `~`, the `=`, or the whole expression (but not simply the colon); the `EQUALS` option is also respected.

If the option `MAGIC_EQUAL_SUBST` is set, any unquoted shell argument in the form `identifier=expression` becomes eligible for file expansion as described in the previous paragraph. Quoting the first `=` also inhibits this.

## Process Substitution

Each command argument of the form `<(list)`, `>(list)` or `=(list)` is subject to process substitution. In the case of the `<` and `>` forms, the shell will run process list asynchronously, connected to a named pipe (FIFO). The name of this pipe will become the argument to the command. If the form with `>` is selected then writing to this file will provide input for list. If `<` is used, then the file passed as an argument will be a named pipe connected to the output of the list process. For example,

`paste <(cut -f1 file1) <(cut -f3 file2) | tee >(process1) >(process2) >/dev/null`

`cut`s fields 1 and 3 from the files file1 and file2 respectively, `paste`s the results together, and sends it to the processes process1 and process2. Note that the file, which is passed as an argument to the command, is a system pipe so programs that expect to `lseek(2)` on the file will not work. Also note that the previous example can be more compactly and efficiently written as:

`paste <(cut -f1 file1) <(cut -f3 file2) >>(process1) >>(process2)`

The shell uses pipes instead of FIFOs to implement the latter two process substitutions in the above example.

If `=` is used, then the file passed as an argument will be the name of a temporary file containing the output of the list process. This may be used instead of the `<` form for a program that expects to `lseek(2)` on the input file.

## Parameter Expansion

The character `$` is used to introduce parameter expansions. See section Parameters, for a description of parameters. In the expansions discussed below that require a pattern, the form of the pattern is the same as that used for filename generation; See section Filename Generation.

`${name}`
  The value, if any, of the parameter name is substituted. The braces are required if name is followed by a letter, digit, or underscore that is not to be interpreted as part of its name. If name is an array parameter, then the values of each element of name is substituted, one element per word. Otherwise, the expansion results in one word only; no word splitting is done on the result.
`${+name}`
  If name is the name of a set parameter, `1` is substituted, otherwise `0` is substituted.
`${name:-word}`
  If name is set and is non-null then substitute its value; otherwise substitute word. If name is missing, substitute word.
`${name:=word}`
  If name is unset or is null then set it to word; the value of the parameter is then substituted.
`${name::=word}`
  Set name to word; the value of the parameter is then substituted.
`${name:?word}`
  If name is set and is non-null, then substitute its value; otherwise, print word and exit from the shell. If word is omitted, then a standard message is printed.
`${name:+word}`
  If name is set and is non-null then substitute word; otherwise substitute nothing.
`${name#pattern}`
`${name##pattern}`
  If the pattern matches the beginning of the value of name, then substitute the value of name with the matched portion deleted; otherwise, just substitute the value of name. In the first form, the smallest matching pattern is preferred; in the second form, the largest matching pattern is preferred. If name is an array and the substitution is not quoted or the `@` flag or the `name[@]` syntax is used, matching is performed on each array elements separately.
`${name%pattern}`
`${name%%pattern}`
  If the pattern matches the end of the value of name, then substitute the value of name with the matched portion deleted; otherwise, just substitute the value of name. In the first form, the smallest matching pattern is preferred; in the second form, the largest matching pattern is preferred. If name is an array and the substitution is not quoted or the `@` flag or the `name[@]` syntax is used, matching is performed on each array elements separately.
`${name:#pattern}`
  If the pattern matches the value of name, then substitute the empty string; otherwise, just substitute the value of name. If name is an array and the substitution is not quoted or the `@` flag or the `name[@]` syntax is used, matching is performed on each array elements separately, and the matched array elements are removed (use the `M` flag to remove the non-matched elements).
`${#spec}`
  If spec is one of the above substitutions, substitute the length in characters of the result instead of the result itself. If spec is an array expression, substitute the number of elements of the result.
`${^spec}`
  Turn on the value of the `RC_EXPAND_PARAM` option for the evaluation of spec; if the `^` is doubled, turn it off. When this option is set, array expansions of the form ``foo${xx}bar'`, where the parameter ``xx'` is set to ``(a b c)'`, are substituted with ``fooabar foobbar foocbar'` instead of the default ``fooa b cbar'`.
`${=spec}`
  Turn on the value of the `SH_WORD_SPLIT` option for the evaluation of spec; if the `=` is doubled, turn it off. When this option is set, parameter values are split into separate words using `IFS` as a delimiter before substitution. This is done by default in most other shells.
`${~spec}`
  Turn on the value of the `GLOB_SUBST` option for the evaluation of spec; if the `~` is doubled, turn it off. When this option is set, any pattern characters resulting from the substitution become eligible for file expansion and filename generation.

If the colon is omitted from one of the above expressions containing a colon, then the shell only checks whether name is set or not, not whether it is null.

If a `${...}` type parameter expression or a `$(...)` type command substitution is used in place of name above, it is substituted first and the result is used as it were the value of name.

If the opening brace is directly followed by an opening parenthesis the string up to the matching closing parenthesis will be taken as a list of flags. Where arguments are valid, any character, or the matching pairs `(...)`, `{...}`, `[...]`, or `<...>`, may be used in place of the colon as delimiters. The following flags are supported:

`A`
  Create an array parameter with `${...:=...}` or `${...::=...}`. Assignment is made before sorting or padding.
`@`
  In double quotes, array elements are put into separate words. Eg. `${(@)foo}` is equivalent to `${foo[@]}` and `${(@)foo[1,2]}` is the same as `$foo[1]` `$foo[2]`.
`e`
  Perform parameter expansion, command substitution and arithmetic expansion on the result. Such expansions can be nested but too deep recursion may have unpredictable effects.
`o`
  Sort the resulting words in ascending order.
`O`
  Sort the resulting words in descending order.
`i`
  With `o` or `O`, makes the sort case-insensitive.
`L`
  Converts all letters in the result to lowercase.
`U`
  Converts all letters in the result to uppercase.
`C`
  Capitalizes the resulting words
`c`
  With `${#name}`, count the total number of characters in an array, as if the elements were concatenated with spaces between them.
`w`
  With `${#name}`, count words in arrays or strings; the `s` flag may be used to set a word delimiter.
`W`
  Similar to `w` with the difference that empty words between repeated delimiters are also counted.
`p`
  Recognize the same escape sequences as the `print` builtin in string arguments to subsequent flags.
`l:expr::string1::string2:`
  Pad the resulting words on the left. Each word will be truncated if required and placed in a field expr characters wide. The space to the left will be filled with string1 (concatenated as often as needed), or spaces if string1 is not given. If both string1 and string2 are given, this string will be placed exactly once directly to the left of the resulting word.
`r:expr::string1::string2:`
  As `l`, but pad the words on the right.
`j:string:`
  Join the words or arrays together using string as a separator. Note that this occurs before word splitting by the `SH_WORD_SPLIT` option.
`F`
  Join the words of arrays together using newline as a separator. This is a shorthand for `pj:\n:`.
`s:string:`
  Force word splitting (see the option `SH_WORD_SPLIT`) at the separator string. Splitting only occurs in places where an array value is valid.
`f`
  Split the result of the expansion to lines. This is a shorthand for `ps:\n:`.
`S`
  (This and all remaining flags are used with the `${...#...}` and `${...%...}` forms). Search substrings as well as beginnings or ends.
`I:expr:`
  Search the expr'th match (where expr evaluates to a number).
`M`
  Include the matched portion in the result.
`R`
  Include the unmatched portion in the result (the Rest).
`B`
  Include the index of the beginning of the match in the result.
`E`
  Include the index of the end of the match in the result.
`N`
  Include the length of the match in the result.

## Command Substitution

A command enclosed in parentheses preceded by a dollar sign, like so: `$(...)` or quoted with grave accents: ``...`` is replaced with its standard output, with any trailing newlines deleted. If the substitution is not enclosed in double quotes, the output is broken into words using the `IFS` parameter. The substitution `$(cat foo)` may be replaced by the equivalent but faster `$(<foo)`. In either case, if the option `GLOB_SUBST` is set the output is eligible for filename generation.

## Arithmetic Expansion

A string of the form `$[exp]` is substituted with the value of the arithmetic expression exp. exp is subjected to parameter expansion, command substitution and arithmetic expansion before it is evaluated. See section Arithmetic Evaluation.

## Brace Expansion

A string of the form ``foo{xx,yy,zz}bar'` is expanded to the individual words ``fooxxbar'`, ``fooyybar'`, and ``foozzbar'`. Left-to-right order is preserved. This construct may be nested. Commas may be quoted in order to include them literally in a word.

An expression of the form `{x-y}`, where x and y are single characters, is expanded to every character between x and y, inclusive.

An expression of the form `{n1..n2}`, where n1 and n2 are integers, is expanded to every number between n1 and n2, inclusive. If either number begins with a zero, all the resulting numbers will be padded with leading zeroes to that minimum width. If the numbers are in decreasing order the resulting sequence will also be in decreasing order.

If a brace expression matches none of the above forms, it is left unchanged, unless the `BRACE_CCL` option is set. In that case, it is expanded to a sorted list of the individual characters between the braces, in the manner of a search set. `-` is treated specially as in a search set, but `^` or `!` as the first character is treated normally.

## Filename Generation

If a word contains an unquoted instance of one of the characters `*`, `|`, `<`, `[`, or `?`, it is regarded as a pattern for filename generation, unless the `GLOB` option is unset. If the `EXTENDED_GLOB` option is set, the `^`, `~`, and `#` characters also denote a pattern; otherwise (except for an initial `~`, See section Filename Expansion) they are not treated specially by the shell. The word is replaced with a list of sorted filenames that match the pattern. If no matching pattern is found, the shell gives an error message, unless the `NULL_GLOB` option is set, in which case the word is deleted; or unless the `NOMATCH` option is unset, in which case the word is left unchanged. In filename generation, the character `/` must be matched explicitly; also, a `.` must be matched explicitly at the beginning of a pattern or after a `/`, unless the `GLOB_DOTS` option is set. No filename generation pattern matches the files ``.'` or ``..'`. In other instances of pattern matching, the `/` and `.` are not treated specially.

`*`
  Matches any string, including the null string.
`?`
  Matches any character.
`[...]`
  Matches any of the enclosed characters. Ranges of characters can be specified by separating two characters by a `-`. A `-` or `]` may be matched by including it as the first character in the list.
`[^...]`
`[!...]`
  Like `[...]`, except that it matches any character which is not in the given set.
`<x-y>`
  Matches any number in the range `x` to `y`, inclusive. If `x` is omitted, the number must be less than or equal to `y`. If `y` is omitted, the number must be greater than or equal to `x`. A pattern of the form `<->` matches any number.
`^x`
  Matches anything except the pattern `x`.
`x|y`
  Matches either `x` or `y`.
`x#`
  Matches zero or more occurrences of the pattern `x`.
`x##`
  Matches one or more occurrences of the pattern `x`.

Parentheses may be used for grouping. Note that the `|` character must be within parentheses, so that the lexical analyzer does not think it is a pipe character. Also note that `/` has a higher precedence than `^`; that is:

`ls ^foo/bar`

will search directories in ``.'` except ``./foo'` for a file named ``bar'`.

A pathname component of the form `(foo/)#` matches a path consisting of zero or more directories matching the pattern foo. As a shorthand, `**/` is equivalent to `(*/)#`. Thus:

`ls (*/)#bar`

or

`ls **/bar`

does a recursive directory search for files named bar, not following symbolic links. For this you can use the form `***/`.

If used for filename generation, a pattern may contain an exclusion specifier. Such patterns are of the form `pat1~pat2`. This pattern will generate all files matching pat1, but which do not match pat2. For example, ``*.c~lex.c'` will match all files ending in ``.c'`, except the file ``lex.c'`. This may appear inside parentheses. Note that `~` has higher precedence than `|`, so that ``pat1|pat2~pat3'` matches any time that pat1 matches, or if pat2 matches while pat3 does not. Note also that any `/` characters are not treated specially in the exclusion specifier, so that a `*` will match multiple path segments if they appear in the pattern to the left of the `~`.

Patterns used for filename generation may also end in a list of qualifiers enclosed in parentheses. The qualifiers specify which filenames that otherwise match the given pattern will be inserted in the argument list. A qualifier may be any one of the following:

`/`
  Directories
`.`
  Plain files
`@`
  Symbolic links
`=`
  Sockets
`p`
  Named pipes (FIFOs)
`*`
  Executable plain files (0100)
`%`
  Device files (character or block special)
`%b`
  Block special files
`%c`
  Character special files
`r`
  owner-readable files (0400)
`w`
  owner-writable files (0200)
`x`
  owner-executable files (0100)
`A`
  group-readable files (0040)
`I`
  group-writable files (0020)
`E`
  group-executable files (0010)
`R`
  world-readable files (0004)
`W`
  world-writable files (0002)
`X`
  world-executable files (0001)
`s`
  Setuid files (04000)
`S`
  Setgid files (02000)
`t`
  files with the sticky bit (01000)
`ddev`
  Files on the device dev
`l[+|-]ct`
  Files having a link count less than `ct (-)`, greater than `ct (+)`, or is equal to `ct`.
`U`
  Files owned by the effective user id.
`G`
  Files owned by the effective group id.
`uid`
  Files owned by user id if id is a number. If not, the character after the `u` will be used as a separator and the string between it and the next matching separator (`(`, `[`, `{`, and `<` match `)`, `]`, `}`, and `>` respectively; any other character matches itself) will be taken as a user name and translated into the corresponding user id (e.g. `u:foo:` or `u[foo]` for user `foo`).
`gid`
  Like `uid` but with group ids or names.
`a[Mwhm][-|+]n`
  Files accessed exactly n days ago. Files accessed within the last n days are selected using a negative value for n (``-n'`). Files accessed more than n days ago are selected by a positive n value (`+n`). Optional unit specifiers `M`, `w`, `h`, or `m` (e.g. `ah5`) cause the check to be performed with months (of 30 days), weeks, hours, or minutes instead of days, respectively. For instance, `echo *(ah-5)` would echo files accessed within the last five hours.
`m[Mwhm][-|+]n`
  Like the file access qualifier, except that it uses the file modification time.
`c[Mwhm][-|+]n`
  Like the file access qualifier, except that it uses the file inode change time.
`L[+-]n`
  Files less than n bytes (`-`), more than n bytes (`+`), or exactly n bytes in length. If this flag is directly followed by a `k` (`K`), `m` (`M`), or `p` (`P`) (e.g. `Lk+50`) the check is performed with kilobytes, megabytes, or blocks (of 512 bytes) instead.
`^`
  Negates all qualifiers following it.
`-`
  Toggles between making the qualifiers work on symbolic links (the default), and the files they point to.
`M`
  Sets the `MARK_DIRS` option for the current pattern.
`T`
  Appends a trailing qualifier mark to the file names, analogous to the `LIST_TYPES`, for the current pattern (overrides `M`).
`N`
  Sets the `NULL_GLOB` option for the current pattern.
`D`
  Sets the `GLOB_DOTS` option for the current pattern.

More than one of these lists can be combined, separated by commas; the whole list matches if at least one of the sublists matches (they are `or`'ed, the qualifiers in the sublists are `and`'ed).

If a `:` appears in a qualifier list, the remainder of the expression in parentheses is interpreted as a modifier (See section Modifiers). Note that each modifier must be introduced by a separate `:`. Note also that the result after modification does not have to be an existing file. The name of any existing file can be followed by a modifier of the form `(:...)` even if no filename generation is performed.

Thus:

`ls *(-/)`

lists all directories and symbolic links that point to directories, and

`ls *(%W)`

lists all world-writable device files in the current directory, and

`ls *(W,X)`

lists all files in the current directory that are world-writable or world-executable, and

`echo /tmp/foo*(u0^@:t)`

outputs the basename of all root-owned files beginning with the string `foo` in ``/tmp'`, ignoring symlinks, and

`ls *.*~(lex|parse).[ch](^D^l1)`

lists all files having a link count of one whose names contain a dot (but not those starting with a dot, since `GLOB_DOTS` is explicitly switched off) except for ``lex.c'`, ``lex.h'`, ``parse.c'`, and ``parse.h'`.

## History Expansion

History substitution allows you to use words from previous command lines in the command line you are typing. This simplifies spelling corrections and the repetition of complicated commands or arguments. Command lines are saved in the history list, the size of which is controlled by the `HISTSIZE` variable. The most recent command is retained in any case. A history substitution begins with the first character of the `histchars` parameter which is `!` by default and may occur anywhere on the command line; history substitutions do not nest. The `!` can be escaped with `\` or can be enclosed between a pair of single quotes (`''`) to suppress its special meaning. Double quotes will not work for this.

Input lines containing history substitutions are echoed on the terminal after being expanded, but before any other substitutions take place or the command gets executed.

### Event Designators

An event designator is a reference to a command-line entry in the history list.

`!`
  Start a history substitution, except when followed by a blank, newline, `=`, or `(`.
`!!`
  Refer to the previous command. By itself, this substitution repeats the previous command.
`!n`
  Refer to command-line n.
`!-n`
  Refer to the current command-line minus n.
`!str`
  Refer to the most recent command starting with str.
`!?str[?]`
  Refer to the most recent command containing str.
`!#`
  Refer to the current command line typed in so far. The line is treated as if it were complete up to and including the word before the one with the `!#` reference.
`!{...}`
  Insulate a history reference from adjacent characters (if necessary).

### Word Designators

A word designator indicates which word or words of a given command line will be included in a history reference. A `:` separates the event specification from the word designator. It can be omitted if the word designator begins with a `^`, `$`, `*`, `-` or `%`. Word designators include:

`0`
  The first input word (command).
`n`
  The n'th argument.
`^`
  The first argument, that is, `1`.
`$`
  The last argument.
`%`
  The word matched by (the most recent) `?str` search.
`x-y`
  A range of words; ``-y'` abbreviates `0-y`.
`*`
  All the arguments, or a null value if there is just one word in the event.
`x*`
  Abbreviates `x-$`.
`x-`
  Like `x*` but omitting word `$`.

Note that a `%` word designator will only work when used as `!%`, `!:%`, `!?str?:%` and only when used after a `!?` substitution. Anything else will result in an error, although the error may not be the most obvious one.

### Modifiers

After the optional word designator, you can add a sequence of one or more of the following modifiers, each preceded by a `:`. These modifiers also work on the result of filename and parameter expansion.

`h`
  Remove a trailing pathname component, leaving the head.
`r`
  Remove a trailing suffix of the form `.xxx`, leaving the basename.
`e`
  Remove all but the suffix.
`t`
  Remove all leading pathname components, leaving the tail.
`&`
  Repeat the previous substitution.
`g`
  Apply the change to the first occurrence of a match in each word, by prefixing the above (for example, `g&`).
`p`
  Print the new command but do not execute it.
`q`
  Quote the substituted words, escaping further substitutions.
`x`
  Like `q`, but break into words at each blank.
`l`
  Convert the words to all lowercase.
`u`
  Convert the words to all uppercase.
`f`
  Repeats the immediately (without a colon) following modifier until the resulting word doesn't change any more. This and the following `F`, `w` and `W` modifier only work with parameter and filename expansion.
`F:expr:`
  Like `f`, but repeats only `n` times if the expression expr evaluates to `n`. Any character can be used instead of the `:`, if any of `(`, `[`, or `{` is used as the opening delimiter the second one has to be `)`, `]`, or `}` respectively.
`w`
  Makes the immediately following modifier work on each word in the string.
`W:sep:`
  Like `w` but words are considered to be the parts of the string that are separated by sep. Any character can be used instead of the `:`, opening parentheses are handled specially, see above.
`s/l/r[/]`
  Substitute r for l.

Unless preceded by a `g`, the substitution is done only for the first string that matches `l`.

The left-hand side of substitutions are not regular expressions, but character strings. Any character can be used as the delimiter in place of `/`. A backslash quotes the delimiter character. The character `&`, in the right hand side, is replaced by the text from the left-hand-side. The `&` can be quoted with a backslash. A null `l` uses the previous string either from a `l` or from a contextual scan string `s` from `!?s`. You can omit the rightmost delimiter if a newline immediately follows `r`; the right-most `?` in a context scan can similarly be omitted.

By default, a history reference with no event specification refers to the same line as the last history reference on that command line, unless it is the first history reference in a command. In that case, a history reference with no event specification always refers to the previous command. However, if the option `CSH_JUNKIE_HISTORY` is set, then history reference with no event specification will always refer to the previous command. For example, `!!:1` will always refer to the first word of the previous command and `!!$` will always refer to the last word of the previous command. And with `CSH_JUNKIE_HISTORY` set, then `!:1` and `!$` will function in the same manner as `!!:1` and `!!$`, respectively. However, if `CSH_JUNKIE_HISTORY` is unset, then `!:1` and `!$` will refer to the first and last words respectively, of the last command referenced on the current command line. However, if they are the first history reference on the command line, then they refer to the previous command.

The character sequence `^foo^bar` repeats the last command, replacing the string foo with the string bar.

If the shell encounters the character sequence `!"` in the input, the history mechanism is temporarily disabled until the current list is fully parsed. The `!"` is removed from the input, and any subsequent `!` characters have no special significance.

A less convenient but more comprehensible form of command history support is provided by the `fc` builtin (see section Shell Builtin Commands).

Go to the first, previous, next, last section, table of contents.