| 1 | ---
|
| 2 | default_highlighter: oils-sh
|
| 3 | ---
|
| 4 |
|
| 5 | YSH FAQ
|
| 6 | =======
|
| 7 |
|
| 8 | Here are some common questions about [YSH]($xref). Many of the answers boil
|
| 9 | down to the fact that YSH is a **smooth upgrade** from [bash]($xref).
|
| 10 |
|
| 11 | Old and new constructs exist side-by-side. New constructs have fewer
|
| 12 | "gotchas".
|
| 13 |
|
| 14 | <!-- cmark.py expands this -->
|
| 15 | <div id="toc">
|
| 16 | </div>
|
| 17 |
|
| 18 | ## Can I freely mix OSH and YSH code?
|
| 19 |
|
| 20 | *If YSH is equivalent to OSH with `shopt --set ysh:all`, it seems like this
|
| 21 | could be possible?*
|
| 22 |
|
| 23 | Currently we recommend only combining OSH and YSH when they're in **different
|
| 24 | processes**.
|
| 25 |
|
| 26 | While you may be able to source OSH from YSH like this:
|
| 27 |
|
| 28 | ```
|
| 29 | shopt --unset ysh:all {
|
| 30 | source lib.osh
|
| 31 | }
|
| 32 | ```
|
| 33 |
|
| 34 | and perhaps source YSH from OSH like this:
|
| 35 |
|
| 36 | ```
|
| 37 | shopt --set ysh:all
|
| 38 | source lib.ysh
|
| 39 | ```
|
| 40 |
|
| 41 | ... actually calling shell functions and procs is **a problem**. This is
|
| 42 | because the `shopt` options are **global**, and you can end up with "mixed
|
| 43 | stacks".
|
| 44 |
|
| 45 | For example, consider a call stack that looks like :
|
| 46 |
|
| 47 | ```
|
| 48 | OSH -> YSH -> OSH -> YSH
|
| 49 | ```
|
| 50 |
|
| 51 | Reasoning about code mixed this way is difficult.
|
| 52 |
|
| 53 | Instead, you may want limited mixing, where you "shell out" to OSH from YSH, or
|
| 54 | vice versa. This is similar to shelling out to `awk` from `sh`, for example.
|
| 55 |
|
| 56 | (The [$0 Dispatch Pattern]($xref:0-dispatch) may be useful.)
|
| 57 |
|
| 58 | ## What's the difference `myvar`, `$myvar`, and `"$myvar"` ?
|
| 59 |
|
| 60 | YSH is more like Python/JavaScript rather than PHP/Perl, so it doesn't use the
|
| 61 | `$` sigil as much.
|
| 62 |
|
| 63 | Never use `$` on the left-hand side:
|
| 64 |
|
| 65 | var mystr = "foo" # not var $mystr
|
| 66 |
|
| 67 | Use `$` to **substitute** vars into commands:
|
| 68 |
|
| 69 | echo $mystr
|
| 70 | echo $mystr/subdir # no quotes in commands
|
| 71 |
|
| 72 | or quoted strings:
|
| 73 |
|
| 74 | echo "$mystr/subdir"
|
| 75 | var x = "$mystr/subdir"
|
| 76 |
|
| 77 | Rarely use `$` on the right-hand side:
|
| 78 |
|
| 79 | var x = mystr # preferred
|
| 80 | var x = $mystr # ILLEGAL -- use remove $
|
| 81 | var x = ${mystr:-} # occasionally useful
|
| 82 |
|
| 83 | var x = $? # allowed
|
| 84 |
|
| 85 | See [Command vs. Expression Mode](command-vs-expression-mode.html) for more
|
| 86 | details.
|
| 87 |
|
| 88 | ## How do I write `~/src` or `~bob/git` in a YSH assignment?
|
| 89 |
|
| 90 | This should cover 80% of cases:
|
| 91 |
|
| 92 | var path = "$HOME/src" # equivalent to ~/src
|
| 93 |
|
| 94 | The old shell style will cover the remaining cases:
|
| 95 |
|
| 96 | declare path=~/src
|
| 97 | readonly other=~bob/git
|
| 98 |
|
| 99 | ---
|
| 100 |
|
| 101 | This is only in issue in *expressions*. The traditional shell idioms work in
|
| 102 | *command* mode:
|
| 103 |
|
| 104 | echo ~/src ~bob/git
|
| 105 | # => /home/alice/src /home/bob/git
|
| 106 |
|
| 107 | The underlying design issue is that the YSH expression `~bob` looks like a
|
| 108 | unary operator and a variable, not some kind of string substitution.
|
| 109 |
|
| 110 | Also, quoted `"~"` is a literal tilde, and shells disagree on what `~""` means.
|
| 111 | The rules are subtle, so we avoid inventing new ones.
|
| 112 |
|
| 113 | <!--
|
| 114 | TODO: I want the ${ ~/src } syntax though it's complicated by ksh command sub
|
| 115 | -->
|
| 116 |
|
| 117 | ## How do I write the equivalent of `echo -e` or `echo -n`?
|
| 118 |
|
| 119 | To echo special characters denoted by backslash escapes, use a
|
| 120 | statically-parsed string literal, not `echo -e`:
|
| 121 |
|
| 122 | echo u'tab \t newline \n' # YES: J8 style string is recommended in YSH
|
| 123 | echo $'tab \t newline \n' # bash-style string is also accepted
|
| 124 |
|
| 125 | These styles don't work in YSH:
|
| 126 |
|
| 127 | echo -e "tab \\t newline \\n" # NO: -e is printed literally
|
| 128 | echo -e "tab \t newline \n" # Error: Invalid char escape
|
| 129 |
|
| 130 | To omit the trailing newline, use the `write` builtin:
|
| 131 |
|
| 132 | write -n -- $prefix # YES
|
| 133 | write --end '' -- $prefix # synonym
|
| 134 |
|
| 135 | echo -n $prefix # NO: -n is printed literally
|
| 136 |
|
| 137 | ### Why Were `-e` and `-n` Removed?
|
| 138 |
|
| 139 | The idioms with `u''` and `write` are more powerful and consistent.
|
| 140 |
|
| 141 | Moreover, shell's `echo` is the *only* builtin that doesn't accept `--` to stop
|
| 142 | flag processing.
|
| 143 |
|
| 144 | That is, `echo "$flag"` always has a few bugs: when `$flag` is `-e`, `-n`,
|
| 145 | `-en`, or `-ne`. There's **no** way to fix this bug in POSIX shell.
|
| 146 |
|
| 147 | So portable shell scripts use:
|
| 148 |
|
| 149 | printf '%s\n' "$x" # print $x "unmolested" in POSIX shell
|
| 150 |
|
| 151 | We could have chosen to respect `echo -- $x`, but YSH already has:
|
| 152 |
|
| 153 | write -- $x # print $x "unmolested" in YSH
|
| 154 |
|
| 155 | That means YSH has:
|
| 156 |
|
| 157 | echo $x # an even shorter way
|
| 158 |
|
| 159 | So `echo` is technically superfluous in YSH, but it's also short, familiar, and
|
| 160 | correct.
|
| 161 |
|
| 162 | YSH isn't intended to be compatible with POSIX shell; only OSH is.
|
| 163 |
|
| 164 | ### How do I write a string literal with both `$myvar` and `\n`?
|
| 165 |
|
| 166 | In YSH, either use `$[ \n ]` inside a double-quoted string:
|
| 167 |
|
| 168 | $ echo "$myvar $[ \n ] two" # expression sub wraps \n
|
| 169 | value_of_myvar
|
| 170 | two
|
| 171 |
|
| 172 | Or use the concatenation operator `++` with two styles of string literal:
|
| 173 |
|
| 174 | echo $[u'newline \n' ++ " $year/$month/$day"]
|
| 175 |
|
| 176 | This POSIX shell behavior is probably not what you want:
|
| 177 |
|
| 178 | $ echo "\n"
|
| 179 | \n # not a newline!
|
| 180 |
|
| 181 | ### How do I find all the `echo` invocations I need to change when using YSH?
|
| 182 |
|
| 183 | A search like this can statically find most usages:
|
| 184 |
|
| 185 | $ egrep -n 'echo (-e|-n|-en|-ne)' *.sh
|
| 186 | test/syscall.sh:58: echo -n hi
|
| 187 | test/syscall.sh:76: echo -e '\t'
|
| 188 |
|
| 189 | ## What's the difference between `$(dirname $x)` and `$[len(x)]` ?
|
| 190 |
|
| 191 | Superficially, both of these syntaxes take an argument `x` and return a
|
| 192 | string. But they are different:
|
| 193 |
|
| 194 | - `$(dirname $x)` is a shell command substitution that returns a string, and
|
| 195 | **starts another process**.
|
| 196 | - `$[len(x)]` is an expression sub containing a function call expression.
|
| 197 | - It doesn't need to start a process.
|
| 198 | - Note that `len(x)` evaluates to an integer, and `$[len(x)]` converts it to
|
| 199 | a string.
|
| 200 |
|
| 201 | <!--
|
| 202 | (Note: builtin subs like `${.myproc $x}` are meant to eliminate process
|
| 203 | overhead, but they're not yet implemented.)
|
| 204 | -->
|
| 205 |
|
| 206 | ## Why doesn't a raw string work here: `${array[r'\']}` ?
|
| 207 |
|
| 208 | This boils down to the difference between OSH and YSH, and not being able to
|
| 209 | mix the two. Though they look similar, `${array[i]}` syntax (with braces) is
|
| 210 | fundamentally different than `$[array[i]]` syntax (with brackets).
|
| 211 |
|
| 212 | - OSH supports `${array[i]}`.
|
| 213 | - The index is legacy/deprecated shell arithmetic like `${array[i++]}` or
|
| 214 | `${assoc["$key"]}`.
|
| 215 | - The index **cannot** be a raw string like `r'\'`.
|
| 216 | - YSH supports both, but [expression substitution][expr-sub] syntax
|
| 217 | `$[array[i]]` is preferred.
|
| 218 | - It accepts YSH expressions like `$[array[i + 1]` or `$[mydict[key]]`.
|
| 219 | - A raw string like `r'\'` is a valid key, e.g. `$[mydict[r'\']]`.
|
| 220 |
|
| 221 | [expr-sub]: ref/chap-expr-lang.html#expr-sub
|
| 222 |
|
| 223 | Of course, YSH style is preferred when compatibility isn't an issue.
|
| 224 |
|
| 225 | No:
|
| 226 |
|
| 227 | echo ${array[r'\']}
|
| 228 |
|
| 229 | Yes:
|
| 230 |
|
| 231 | echo $[array[r'\']]
|
| 232 |
|
| 233 | A similar issue exists with arithmetic.
|
| 234 |
|
| 235 | Old:
|
| 236 |
|
| 237 | echo $((1 + 2)) # shell arithmetic
|
| 238 |
|
| 239 | New:
|
| 240 |
|
| 241 | echo $[1 + 2] # YSH expression
|
| 242 |
|
| 243 | <!--
|
| 244 |
|
| 245 | ## Why doesn't the ternary operator work here: `${array[0 if cond else 5]}`?
|
| 246 |
|
| 247 | The issue is the same as above. YSH expression are allowed within `$[]` but
|
| 248 | not `${}`.
|
| 249 |
|
| 250 | -->
|
| 251 |
|
| 252 | ## How do I combine conditional commands and expressions: `if (myvar)` and `if test -f`?
|
| 253 |
|
| 254 | You can use the `--true` and `--false` flags to the [YSH test][ysh-test]
|
| 255 | builtin:
|
| 256 |
|
| 257 | if test --true $[myvar] && test --file x {
|
| 258 | echo ok
|
| 259 | }
|
| 260 |
|
| 261 | They test if their argument is literally the string `"true"` or `"false"`.
|
| 262 |
|
| 263 | This works because the boolean `true` *stringifies* to `"true"`, and likewise
|
| 264 | with `false`.
|
| 265 |
|
| 266 | [ysh-test]: ref/chap-builtin-cmd.html#ysh-test
|
| 267 |
|
| 268 |
|
| 269 | ## Why do I lose the value of `p` in `myproc (&p) | grep foo`?
|
| 270 |
|
| 271 | In a pipeline, most components are **forked**. This means that `myproc (&p)`
|
| 272 | runs in a different process from the main shell.
|
| 273 |
|
| 274 | The main shell can't see the memory of a subshell.
|
| 275 |
|
| 276 | ---
|
| 277 |
|
| 278 | In general, you have to restructure your code to avoid this. You could use a proc with multiple outputs:
|
| 279 |
|
| 280 | myproc (&p, &grepped_output)
|
| 281 |
|
| 282 | Or you could use a function:
|
| 283 |
|
| 284 | var out1, out2 = myfunc(io)
|
| 285 |
|
| 286 | ---
|
| 287 |
|
| 288 | [The Unix Shell Process Model - When Are Processes
|
| 289 | Created?](process-model.html) may help.
|
| 290 |
|
| 291 | This issue is similar to the `shopt -s lastpipe` issue:
|
| 292 |
|
| 293 | $ bash -c 'echo hi | read x; echo x=$x'
|
| 294 | x=
|
| 295 |
|
| 296 | $ zsh -c 'echo hi | read x; echo x=$x'
|
| 297 | x=hi
|
| 298 |
|
| 299 | In bash, `read` runs in a subshell, but in `zsh` and OSH, it runs in the main
|
| 300 | shell.
|
| 301 |
|
| 302 | ## Why are `Dict` and `Obj` different types?
|
| 303 |
|
| 304 | *JavaScript has a single Object type, while Python has separate dicts and
|
| 305 | objects.*
|
| 306 |
|
| 307 | In YSH, we draw a line between data and code.
|
| 308 |
|
| 309 | - A `Dict` is pure **data**, and may correspond to JSON from untrusted sources.
|
| 310 | - An `Obj` bundles both data and **code**, and can't be serialized by default.
|
| 311 |
|
| 312 | You can create an `Obj` from a `Dict` with the `Obj` constructor. Conversely,
|
| 313 | you can get the first Dict in an object with [first(myobj)][first].
|
| 314 |
|
| 315 | There is no special `__proto__` or `prototype` name, which reduces the
|
| 316 | likelihood of "prototype pollution" vulnerabilities.
|
| 317 |
|
| 318 | ---
|
| 319 |
|
| 320 | This is essentially the [Interior vs. Exterior][interior-exterior] distinction:
|
| 321 | An Obj lives inside the shell process, while a Dict may come from outside the
|
| 322 | process (user input).
|
| 323 |
|
| 324 | [first]: ref/chap-builtin-func.html#first
|
| 325 | [interior-exterior]: https://www.oilshell.org/blog/2023/06/ysh-design.html
|
| 326 |
|
| 327 | ## Why are `Command` and `Proc` different types?
|
| 328 |
|
| 329 | *Could a `Command` be a `Proc` with no arguments? Similarly, could an `Expr` be a
|
| 330 | `Func` with no arguments?*
|
| 331 |
|
| 332 | Procs and Funcs both push a new stack frame, and bind arguments to declared
|
| 333 | parameters.
|
| 334 |
|
| 335 | On the other hand, `Command` and `Expr` are more "raw" and flexible:
|
| 336 |
|
| 337 | - They can be evaluated in different stack frames —
|
| 338 | e.g. `io->eval(b, in_captured_frame)`
|
| 339 | - They can have "undeclared" variable bindings —
|
| 340 | e.g. `io->eval(b, vars={x: 42})`.
|
| 341 |
|
| 342 | In other words, they're low-level, reflective types that allow users to create
|
| 343 | expressive APIs, like:
|
| 344 |
|
| 345 | cd /tmp { # Command literal (block)
|
| 346 | ls -l
|
| 347 | }
|
| 348 | my-table | where [size > 3] # Expr literal
|
| 349 |
|
| 350 | ---
|
| 351 |
|
| 352 | Another way to think about it: we could have removed procs from the core YSH
|
| 353 | language, and implemented them in terms of command blocks and `io->eval()`.
|
| 354 | But that seems too low-level!
|
| 355 |
|
| 356 | ## Related
|
| 357 |
|
| 358 | - [YSH FAQ]($wiki) on the wiki has more answers. We may be migrate some of
|
| 359 | them here.
|
| 360 |
|