OILS / doc / ref / chap-cmd-lang.md View on Github | oils.pub

776 lines, 500 significant
1---
2title: Command Language (Oils Reference)
3all_docs_url: ..
4body_css_class: width40
5default_highlighter: oils-sh
6preserve_anchor_case: yes
7---
8
9<div class="doc-ref-header">
10
11[Oils Reference](index.html) &mdash;
12Chapter **Command Language**
13
14</div>
15
16This chapter describes the command language for OSH, and some YSH extensions.
17
18<span class="in-progress">(in progress)</span>
19
20<div id="dense-toc">
21</div>
22
23## Quick Sketch: What's a Command?
24
25OSH:
26
27 print-files() {
28 for name in *.py; do
29 if test -x "$name"; then
30 echo "$name is executable"
31 fi
32 done
33 }
34
35YSH:
36
37 proc print-files {
38 for name in *.py {
39 if test -x $name { # no quotes needed
40 echo "$name is executable"
41 }
42 }
43 }
44
45
46<h2 id="Commands">Commands</h2>
47
48### simple-command
49
50Commands are composed of words. The first word may be the name of
51
521. An alias
531. A builtin command
541. A YSH `proc`
551. A shell "function"
561. An external command
57
58Examples:
59
60 echo hi # a shell builtin doesn't start a process
61 ls /usr/bin ~/src # starts a new process
62 myproc "hello $name"
63 myshellfunc "hello $name"
64 myalias -l
65<!-- TODO: document lookup order -->
66
67Redirects are also allowed in any part of the command:
68
69 echo 'to stderr' >&2
70 echo >&2 'to stderr'
71
72 echo 'to file' > out.txt
73 echo > out.txt 'to file'
74
75### command-lookup-order
76
77The first word in a command can mean many things.
78
79(1) An [alias][] may expand into shell code at **parse** time. This includes
80keywords!
81
82Aliases are **disabled** in YSH (`shopt --unset expand_aliases`).
83
84(2) Shell keywords like `if` have their own **parsing** rules:
85
86 if test -d /; then # OSH style
87 echo yes
88 fi
89
90 if test --dir / { # YSH style
91 echo yes
92 }
93
94(3) The first word of a [simple command][simple-command] is looked up in this
95order:
96
971. Special builtins like `eval`
981. YSH Procs or Shell Functions
991. Normal Builtins like `cd`
1001. External Processes like `/bin/ls`
101
102So special builtins can't be redefined as functions, but normal builtins can.
103
104YSH also adds the concept of a **private** builtin. Private
105builtins like [sleep][] are **not** consulted as the first word:
106
107 sleep 1 # run external command
108 builtin sleep 1 # run private builtin with explicit prefix
109
110[alias]: chap-builtin-cmd.html#alias
111[simple-command]: chap-cmd-lang.html#simple-command
112[sleep]: chap-builtin-cmd.html#sleep
113
114To summarize, OSH commands are looked up in this order:
115
1161. alias
1171. keyword
1181. special builtin
1191. shell function
1201. normal builtin
1211. external
122
123YSH commands are looked up in this order:
124
1251. keyword
1261. special builtin
1271. YSH proc
1281. normal builtin
1291. external
130
131Logically separate:
132
1336. private builtin (not the first word)
134
135Use [invoke --show][invoke] to see different meanings for a given name.
136
137[invoke]: chap-builtin-cmd.html#invoke
138
139### prefix-binding
140
141Bindings are allowed before a simple command:
142
143 PYTHONPATH=. mydir/myscript.py
144
145These bindings set a variable and mark it exported. This binding is usually
146temporary, but when used with certain [special builtins][special], it persists.
147
148[special]: https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html
149
150- Related: [ysh-prefix-binding](ysh-prefix-binding)
151
152### ysh-prefix-binding
153
154YSH prefix bindings look exactly like they do in shell:
155
156 PYTHONPATH=. mydir/myscript.py
157
158However, they temporarily set `ENV.PYTHONPATH`, not `$PYTHONPATH`. This is
159done by adding a new `Dict` to the prototype chain of the `Obj`.
160
161The new `ENV` then becomes the environment of the child processes for the
162command.
163
164In YSH, prefix bindings are always temporary ENV bindings, and they don't
165persist. This is enforced by `shopt --set strict_env_binding`.
166
167- Related: [ENV](chap-special-var.html#ENV), [prefix-binding](chap-cmd-lang.html#prefix-binding)
168
169
170<h3 id="semicolon" class="osh-ysh-topic">semicolon ;</h3>
171
172Run two commands in sequence like this:
173
174 echo one; echo two
175
176or this:
177
178 echo one
179 echo two
180
181<h2 id="Conditional">Conditional</h2>
182
183<h3 id="case" class="osh-topic">case</h3>
184
185Match a string against a series of glob patterns. Execute code in the section
186below the matching pattern.
187
188 path='foo.py'
189 case "$path" in
190 *.py)
191 echo 'python'
192 ;;
193 *.sh)
194 echo 'shell'
195 ;;
196 esac
197
198For bash compatibility, the `;;` terminator can be substituted with either:
199
200- `;&` - fall through to next arm, ignoring the condition
201- `;;&` - fall through to next arm, respecting the condition
202
203<h3 id="if" class="osh-topic">if</h3>
204
205Test if a command exited with status zero (true). If so, execute the
206corresponding block of code.
207
208Shell:
209
210 if test -d foo; then
211 echo 'foo is a directory'
212 elif test -f foo; then
213 echo 'foo is a file'
214 else
215 echo 'neither'
216 fi
217
218YSH:
219
220 if test -d foo {
221 echo 'foo is a directory'
222 } elif test -f foo {
223 echo 'foo is a file'
224 } else {
225 echo 'neither'
226 }
227
228<h3 id="dbracket" class="osh-topic">dbracket [[</h3>
229
230Statically parsed boolean expressions, from bash and other shells:
231
232 x=42
233 if [[ $x -eq 42 ]]; then
234 echo yes
235 fi # => yes
236
237Compare with the [test][] builtin, which is dynamically parsed.
238
239See [bool-expr][] for the expression syntax.
240
241[test]: chap-builtin-cmd.html#test
242[bool-expr]: chap-mini-lang.html#bool-expr
243
244
245<h3 id="bang" class="osh-ysh-topic">bang !</h3>
246
247Invert an exit code:
248
249 if ! test -d /tmp; then
250 echo "No temp directory
251 fi
252
253<h3 id="and" class="osh-ysh-topic">and &amp;&amp;</h3>
254
255 mkdir -p /tmp && cp foo /tmp
256
257<h3 id="or" class="osh-ysh-topic">or ||</h3>
258
259 ls || die "failed"
260
261<h2 id="Iteration">Iteration</h2>
262
263<h3 id="while" class="osh-ysh-topic">while</h3>
264
265POSIX
266
267<h3 id="until" class="osh-topic">until</h3>
268
269POSIX
270
271<h3 id="for" class="osh-ysh-topic">for</h3>
272
273For loops iterate over words.
274
275YSH style:
276
277 var mystr = 'one'
278 var myarray = :| two three |
279
280 for i in $mystr @myarray *.py {
281 echo $i
282 }
283
284
285Shell style:
286
287 local mystr='one'
288 local myarray=(two three)
289
290 for i in "mystr" "${myarray[@]}" *.py; do
291 echo $i
292 done
293
294Both fragments output 3 lines and then Python files on remaining lines.
295
296<h3 id="for-expr-sh" class="osh-topic">for-expr-sh</h3>
297
298A bash/ksh construct:
299
300 for (( i = 0; i < 5; ++i )); do
301 echo $i
302 done
303
304<h2 id="Control Flow">Control Flow</h2>
305
306These are keywords in Oils, not builtins!
307
308### break
309
310Break out of a loop. (Not used for case statements!)
311
312### continue
313
314Continue to the next iteration of a loop.
315
316### return
317
318Return from a function.
319
320### exit
321
322Exit the shell process with the given status:
323
324 exit 2
325
326<h2 id="Grouping">Grouping</h2>
327
328### sh-func
329
330POSIX:
331
332 f() {
333 echo args "$@"
334 }
335 f 1 2 3
336
337### sh-block
338
339POSIX:
340
341 { echo one; echo two; }
342
343The trailing `;` is necessary in OSH, but not YSH. In YSH, `parse_brace` makes
344`}` is more of a special word.
345
346
347### subshell
348
349 ( echo one; echo two )
350
351In YSH, use [forkwait](chap-builtin-cmd.html#forkwait) instead of parentheses.
352
353<h2 id="Concurrency">Concurrency</h2>
354
355### pipe
356
357Pipelines are a traditional POSIX shell construct:
358
359 ls /tmp | grep ssh | sort
360
361Related:
362
363- [`PIPESTATUS`]() in OSH
364- [`_pipeline_status`]() in YSH
365
366[PIPESTATUS]: chap-special-var.html#PIPESTATUS
367[_pipeline_status]: chap-special-var.html#_pipeline_status
368
369<h3 id="ampersand" class="osh-topic">ampersand &amp;</h3>
370
371Start a command as a background job. Don't wait for it to finish, and return
372control to the shell.
373
374The PID of the job is recorded in the `$!` variable.
375
376 sleep 1 &
377 echo pid=$!
378 { echo two; sleep 2 } &
379 wait
380 wait
381
382In YSH, use the [fork][] builtin.
383
384[fork]: chap-builtin-cmd.html#fork
385
386
387<h2 id="Redirects">Redirects</h2>
388
389### redir-file
390
391The operators `>` and `>>` redirect the `stdout` of a process to a disk file.
392The `<` operator redirects `stdin` from a disk file.
393
394---
395
396Examples of redirecting the `stdout` of a command:
397
398 echo foo > out.txt # overwrite out.txt
399 date >> stamp.txt # append to stamp.txt
400
401<!--
402 echo foo >| out.txt # clobber the file even if set -o noclobber
403-->
404
405Redirect to the `stdin` of a command:
406
407 cat < in.txt
408
409Redirects are compatible with POSIX and bash, so they take descriptor numbers
410on the left:
411
412 make 2> stderr.txt # '2>' is valid, but '2 >' is not
413
414Note that the word argument to **file** redirects is evaluated like bash, which
415is different than other arguments to other redirects:
416
417 tar -x -z < Python* # glob must expand to exactly 1 file
418 tar -x -z < $myvar # $myvar is split because it's unquoted
419
420In other words, it's evaluated **as** a sequence of 1 word, which **produces**
421zero to N strings. But redirects are only valid when it produces exactly 1
422string.
423
424(Related: YSH uses `shopt --set simple_word_eval`, which means that globs that
425match nothing evaluate to zero strings, not themselves.)
426
427<!-- They also take a file descriptor on the left -->
428
429
430### redir-desc
431
432Redirect to a file descriptor:
433
434 echo 'to stderr' >&2
435
436<!--
437NOTE: >&2 is just like <&2
438There's no real difference.
439-->
440
441### here-doc
442
443Here documents let you write the `stdin` of a process in the shell program.
444
445Specify a delimiter word (like EOF) after the redir operator (like `<<`).
446
447If it's unquoted, then `$` expansion happens, like a double-quoted string:
448
449 cat <<EOF
450 here doc with $double ${quoted} substitution
451 EOF
452
453If the delimiter is quoted, then `$` expansion does **not** happen, like a
454single-quoted string:
455
456 cat <<'EOF'
457 price is $3.99
458 EOF
459
460Leading tabs can be stripped with the `<<-` operator:
461
462 myfunc() {
463 cat <<-EOF
464 here doc with one tab leading tab stripped
465 EOF
466 }
467
468### here-str
469
470The `<<<` operator means that the argument is a `stdin` string, not a
471chosen delimiter.
472
473 cat <<< 'here string'
474
475The string **plus a newline** is the `stdin` value, which is consistent with
476GNU bash.
477
478### ysh-here-str
479
480You can also use YSH multi-line strings as "here strings". For example:
481
482Double-quoted:
483
484 cat <<< """
485 double
486 quoted = $x
487 """
488
489Single-quoted:
490
491 cat <<< '''
492 price is
493 $3.99
494 '''
495
496J8-style with escapes:
497
498 cat <<< u'''
499 j8 style string price is
500 mu = \u{3bc}
501 '''
502
503In these cases, a trailing newline is **not** added. For example, the first
504example is equivalent to:
505
506 write --end '' -- """
507 double
508 quoted = $x
509 """
510
511## Other Command
512
513<h3 id="dparen" class="osh-topic">dparen ((</h3>
514
515<h3 id="time" class="osh-ysh-topic">time</h3>
516
517 time [-p] pipeline
518
519Measures the time taken by a command / pipeline. It uses the `getrusage()`
520function from `libc`.
521
522Note that time is a KEYWORD, not a builtin!
523
524<!-- Note: bash respects TIMEFORMAT -->
525
526
527## YSH Simple
528
529### typed-arg
530
531Internal commands (procs and builtins) accept typed arguments in parentheses:
532
533 json write (myobj)
534
535Redirects can also appear after the typed args:
536
537 json write (myobj) >out.txt
538
539### lazy-expr-arg
540
541Expressions in brackets like this:
542
543 assert [42 === x]
544
545Are syntactic sugar for:
546
547 assert (^[42 === x])
548
549That is, it's single arg of type `value.Expr`.
550
551Redirects can also appear after the lazy typed args:
552
553 assert [42 === x] >out.txt
554
555- Related: [Expr][] type
556
557[Expr]: chap-type-method.html#Expr
558
559### block-arg
560
561Blocks can be passed to simple commands, either literally:
562
563 cd /tmp {
564 echo $PWD # prints /tmp
565 }
566 echo $PWD
567
568Or as an expression:
569
570 var block = ^(echo $PWD)
571 cd /tmp (; ; block)
572
573Note that `cd` has no typed or named arguments, so the two semicolons are
574preceded by nothing.
575
576When passed to procs, blocks capture the enclosing stack frame:
577
578 var x = 42
579 myproc {
580 # lexical scope is respected
581 echo "x = $x" # x = 42
582 }
583
584---
585
586Redirects can appear after the block arg:
587
588 cd /tmp {
589 echo $PWD # prints /tmp
590 } >out.txt
591
592
593Related:
594
595- [sh-block](#sh-block) in OSH.
596- [Command][] and [CommandFrag][] types.
597
598[Command]: chap-type-method.html#Command
599[CommandFrag]: chap-type-method.html#CommandFrag
600
601## YSH Cond
602
603### ysh-case
604
605Like the shell case statement, the Ysh case statement has **string/glob** patterns.
606
607 var s = 'README.md'
608 case (s) {
609 *.py { echo 'Python' }
610 *.cc | *.h { echo 'C++' }
611 * { echo 'Other' }
612 }
613 # => Other
614
615We also generated it to **typed data** within `()`:
616
617 var x = 43
618 case (x) {
619 (30 + 12) { echo 'the integer 42' }
620 (else) { echo 'neither' }
621 }
622 # => neither
623
624The `else` is a special keyword that matches any value.
625
626 case (s) {
627 / dot* '.md' / { echo 'Markdown' }
628 (else) { echo 'neither' }
629 }
630 # => Markdown
631
632### ysh-if
633
634Like shell, you can use a command:
635
636 if test --file $x {
637 echo "$x is a file"
638 }
639
640You can also use an expression:
641
642 if (x > 0) {
643 echo 'positive'
644 }
645
646## YSH Iter
647
648### ysh-for
649
650#### Words
651
652This is a shell-style loop over "words":
653
654 for word in 'oils' $num_beans {pea,coco}nut {
655 echo $word
656 }
657 # =>
658 # oils
659 # 13
660 # peanut
661 # coconut
662
663You can ask for the loop index with `i,`:
664
665 for i, name in README.md *.py {
666 echo "$i $name"
667 }
668 # => 0 README.md
669 # => 1 foo.py
670
671#### Expressions Over Typed Data
672
673Expressions are enclosed in `()`. You can iterate over a `Range`, `List`,
674`Dict`, or `io.stdin`.
675
676Range:
677
678 for i in (3 ..< 5) { # range operator ..<
679 echo "i = $i"
680 }
681 # =>
682 # i = 3
683 # i = 4
684
685List:
686
687 var foods = ['ale', 'bean']
688 for item in (foods) {
689 echo $item
690 }
691 # =>
692 # ale
693 # bean
694
695---
696
697There are **three** ways of iterating over a `Dict`:
698
699 var mydict = {pea: 42, nut: 10}
700 for key in (mydict) {
701 echo $key
702 }
703 # =>
704 # pea
705 # nut
706
707 for key, value in (mydict) {
708 echo "$key $value"
709 }
710 # =>
711 # pea - 42
712 # nut - 10
713
714 for i, key, value in (mydict) {
715 echo "$i $key $value"
716 }
717 # =>
718 # 0 - pea - 42
719 # 1 - nut - 10
720
721That is, if you ask for two things, you'll get the key and value. If you ask
722for three, you'll also get the index.
723
724(One way to think of it: `for` loops in YSH have the functionality Python's
725`enumerate()`, `items()`, `keys()`, and `values()`.)
726
727---
728
729The `io.stdin` object iterates over lines:
730
731 for line in (io.stdin) {
732 echo $line
733 }
734 # lines are buffered, so it's much faster than `while read --raw-line`
735
736---
737
738(This section is based on [A Tour of YSH](../ysh-tour.html).)
739
740#### Closing Over the Loop Variable
741
742Each iteration of a `for` loop creates a new frame, which may be captured.
743
744 var x = 42 # outside the loop
745 for i in (0 ..< 3) {
746 var j = i + 2
747
748 var expr = ^"$x: i = $i, j = $j" # captures x, i, and j
749
750 my-task {
751 echo "$x: i = $i, j = $j" # also captures x, i, and j
752 }
753 }
754
755#### Mutating Containers in a `for` Loop
756
757- If you append or remove from a `List` while iterating over it, the loop **will** be affected.
758- If you mutate a `Dict` while iterating over it, the loop will **not** be
759 affected.
760
761### ysh-while
762
763You can use an expression as the condition:
764
765 var x = 5
766 while (x < 0) {
767 setvar x -= 1
768 }
769
770You or a command:
771
772 while test -f myfile {
773 echo 'myfile'
774 sleep 1
775 }
776