posix shell tricks, when'd I add others???
This commit is contained in:
parent
b92ca14e52
commit
3915d0d1fc
|
|
@ -5,18 +5,28 @@ there's some very unknown quirky things that you can do with pure posix shell. t
|
|||
[in bash](https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html#index-_005b_005b), `BASH_REMATCH` is an array that corresponds to the groups that are captured in the last used `=~` command. this can be emulated in posix shell by a little-known command called [expr](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/expr.html). **for almost all use cases, expr is superseded by [test](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html) and [arithmetic expansion](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04), which are shell builtins, and you shouldn't use** `expr`, *but* one singular operand in `expr` is unique to it: `:`. take a look:
|
||||
|
||||
```
|
||||
#/bin/sh
|
||||
str="https://example.com/post/120937"
|
||||
if rematch="$(expr -- "$str" : 'https\{0,1\}://\([^/]\{1,\}\)')"; then
|
||||
echo "$rematch"
|
||||
$ cat expr.sh
|
||||
if rematch="$(expr -- "$1" : 'https\{0,1\}://\([^/]\{1,\}\)')"; then
|
||||
echo "$rematch"
|
||||
else
|
||||
echo "not a valid URL!"
|
||||
fi
|
||||
|
||||
$ sh expr.sh https://example.com/post/120937
|
||||
example.com
|
||||
$ sh expr.sh "a random string"
|
||||
not a valid URL!
|
||||
```
|
||||
|
||||
this script will output `example.com`. if `str` didn't match the pattern, then `expr` would return 1, and thus the `if` statement would not run. this allows one to both test if a string matches a regex pattern and to return a part of the string in one call. there are some caveats to doing this, though. the first is that you must use [basic regular expressions](https://en.wikibooks.org/wiki/Regular_Expressions/POSIX_Basic_Regular_Expressions); the second is that the pattern is by default anchored to the start of the line, which can be remedied by putting `.*` at the beginning; and the third is that only the first capturing group will be returned, even if you have more than one
|
||||
as you can see, this allows one to both test if a string matches a regex pattern and to return a part of the string in one call. there are some caveats to doing this, though:
|
||||
|
||||
- you must use [basic regular expressions](https://en.wikibooks.org/wiki/Regular_Expressions/POSIX_Basic_Regular_Expressions), which is much more cumbersome than the more friendly extended regular expressions
|
||||
- the pattern is by default anchored to the start of the line, which can be remedied by putting `.*` at the beginning
|
||||
- only the first capturing group will be returned, even if you have more than one
|
||||
|
||||
if you don't need the rematch at all, then [here are other ways of matching a string against a pattern, regex or not](https://stackoverflow.com/questions/21115121/how-to-test-if-string-matches-a-regex-in-posix-shell-not-bash), and they should usually be preferred over `expr`. `expr` just has one singular very niche use case that shines when its time is right, such as [how I use it in mimix](https://git.unix.dog/yosh/mimix/src/branch/main/xdg-open#L118)
|
||||
|
||||
## eval
|
||||
## eval and escaping
|
||||
there's [a lot of scare about `eval`](https://mywiki.wooledge.org/BashFAQ/048), and for very good reason! it's very powerful yet very dangerous when used in the wrong context
|
||||
|
||||
and that's the thing I want to focus on: **in the wrong context**. I usually find `eval` being referred to as "parsing your code twice", which I think is a bit of a misnomer. for me personally, the [posix definition of `eval`](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_19) makes it easier to understand. once your outside script parses all the expansions, `eval` will use the arguments generated from the expansion as shell code, effectively being the same thing as `sh -c`.
|
||||
|
|
@ -25,4 +35,43 @@ this leads in to how I personally treat `eval`, and how I feel others should tre
|
|||
|
||||
if the users of your script are already expected to have a shell, and if the script does not run with elevated privileges, then `eval` poses no more of a security threat than whatever the user can do in a normal shell they can access, though it can easily still cause headaches to use
|
||||
|
||||
`eval` can also be very useful when building command line arguments from user input, so long as you take great care to escape said input. in posix shell, this is as simple as doing `escaped_input="'$(printf '%s' "$input" | sed "s/'/'\\\\''/g")'"`. this wraps the input in single quotes, where a shell never expands any special characters within except `'`, which are also dealt with by replacing them with `'\''`, as one would in a normal shell. I do this in [agetar](https://git.unix.dog/yosh/agetar/src/branch/master/agetar#L64)
|
||||
`eval` can also be very useful when building command line arguments from user input, so long as you take great care to escape said input. in posix shell, this is as simple as doing `escaped_input="'$(printf '%s' "$input" | sed "s/'/'\\\\''/g")'"`. this wraps the input in single quotes, where a shell never expands any special characters within except `'`, which are also dealt with by replacing them with `'\''`, as one would in a normal shell. in essence, this is doing the exact same thing as `printf %q` from bash! I utilize this for eval command argument building in [agetar](https://git.unix.dog/yosh/agetar/src/branch/master/agetar#L64)
|
||||
|
||||
### a miscellaneous quirk to note
|
||||
process substitution strips trailing newlines, so technically doing `escaped_input="'$(printf '%s' "$input" | sed "s/'/'\\\\''/g")'"` isn't enough. for 99.9999% of intents and purposes, you don't need to worry about stripping trailing newlines at all, but if you somehow need to or want that guarantee of complete and utter safety, replace it with the following: `escaped_input="'$(printf '%s\n' "$input" | sed -e "s/'/'\\\\''/g" -e "s/^$/''/")'"`. this makes sure that there's no trailing newlines and allows one to breathe easy
|
||||
|
||||
## pipeline trick
|
||||
I'm shamelessly reposting most of what is shown [in this excellent github page](https://gist.github.com/izabera/cc9f21e1541d603da66cb28093f46892) by izabera to preserve it somewhere other than github
|
||||
|
||||
if you want to send the stdout of a process to the stdin of *multiple* processes at once, it's simple in bash:
|
||||
|
||||
```
|
||||
gives_output | tee >(needs_input) >(needs_input_2) >/dev/null
|
||||
```
|
||||
|
||||
however, it's much trickier in posix shell. the first thought that comes to many minds (including mine!) is to make fifos for communication with the processes in a background shell:
|
||||
|
||||
```
|
||||
mkfifo fifo1 fifo2
|
||||
needs_input < fifo1 &
|
||||
needs_input_2 < fifo2 &
|
||||
gives_output | tee fifo1 fifo2 >/dev/null
|
||||
```
|
||||
|
||||
however, doing this means a separate process group is created for each individual program, which makes it very messy to clean up if you don't know what you're doing and have to `^C` the process for some reason (I dealt with this when making [`twitch-notify`](https://git.unix.dog/yosh/misc-scripts/src/branch/master/twitch-notify)). to work around this issue, you can instead abuse pipes in a way they sorely weren't intended to be used, just like so:
|
||||
|
||||
```
|
||||
mkfifo fifo1 fifo2
|
||||
gives_output | tee fifo1 fifo2 >/dev/null | \
|
||||
needs_input < fifo1 | needs_input_2 < fifo2
|
||||
```
|
||||
|
||||
this keeps all the processes in one nice little process group, and you don't have to worry about lingering processes when you `^C` while it runs. you can even handle cleaning up the fifos in the pipeline:
|
||||
|
||||
```
|
||||
... | { rm fifo1; needs_input; } < fifo1 | ...
|
||||
```
|
||||
|
||||
this looks dangrous, but it's safe since the shell executes redirection first, then opens the fifo in read blocks until there's a writer
|
||||
|
||||
very cool!
|
||||
|
|
|
|||
|
|
@ -1,3 +0,0 @@
|
|||
<ul>
|
||||
<li>/usr/local/lib in addendums</li>
|
||||
</ul>
|
||||
|
|
@ -1,29 +0,0 @@
|
|||
<?shTITLE="voidlinux|"./head.sh?>
|
||||
|
||||
<body>
|
||||
<?sh./header.sh?>
|
||||
<?sh./sidebar.shtopbottom?>
|
||||
<main>
|
||||
<?shTB_TITLE="voidlinux|xorg"./titlebar.sh?>
|
||||
<h1>voidlinux-xorg</h1>
|
||||
<p>bynow,youshouldstartdoingtherestofthisguideasanormaluserandusing<code>doas</code>or<code>su-</code>toinstallpackagesoreditconfigfilesthatrequireroot</p>
|
||||
<p>I'llmakethisspecificpartbrief.install<code>xorg-minimal</code>,<code>xorg-fonts</code>,yourfavoriteterminalemulator(<code>rxvt-unicode</code>forme),yourfavoritewindowmanager,andthedriversforyourgraphicscard(youcanseewhichpackageyouneedtoinstallwith<code>xbps-query-Rsxf86-video</code>.somethingImuststressisthatIamonlyfamiliarwithxorgthroughthelensof"non-integrating"windowmanagers.thatmeansnooverarchingDElikekde,gnome,etc.ordisplaymanagerlikelightdm,etc.Icannotguaranteeeverythingonthispagewillapplyperfectlytothosekindsofsetups</p>
|
||||
<h3>disablingrootpriviledge</h3>
|
||||
</p>voidlinuxrunsxorgasrootbydefault.Idonotknowwhy.tomyknowledgethereiszerobreakagewithmysetupwhenyouconfigureittonotrunasroot.todothat,create/edit<code>/etc/X11/Xwrapper.config</code>withthefollowing:
|
||||
<codeclass="codeblock">
|
||||
<pre>
|
||||
needs_root_rights=no
|
||||
</pre>
|
||||
</code>
|
||||
<p>that'sit.nowxorgwillnotrunwithrootrightsandlogto<code>~/.local/share/xorg/Xorg.DISP.log</code>.followthe<ahref="https://docs.voidlinux.org/config/graphical-session/xorg.html">manual</a>foraddingyourwindowmanagerto~/.xinitrc,thenuse<code>startx</code>tolaunchanXsession</p>
|
||||
<h2id="local-services">localservices</h2>
|
||||
<p>settinguplocalservicesisincrediblyuseful.however,Idon'tlikethewaythemanualdescribessettingthemup.itseverelylimitswhatservicesyoucanuseasawholeandlikestobefinnickywithstufflikepipewireforme.sohere'showIsetuplocalservices</p>
|
||||
<p>maketwodirectories,<code>~/.local/sv</code>and<code>~/.local/service</code>.thesewillactjustlike<code>/etc/sv</code>and<code>/var/service</code></p>
|
||||
<p>inyourxinitrc,addtheline<code>runsvdir"$HOME/.local/service"&</code><b>before</b>the<code>execyour-window-manager</code>line.now,wheneveryou<code>startx</code>,arunitrunsvprogramwillsuperviseyourlocalservicedirectory,justlikehowitdoesasroot.now,thisdoesn'tallowforservicesinatty,butI'vehonestlyneverneededanylocaluserspecificserviceswhileinattybefore,sothisworksforme</p>
|
||||
<footer>
|
||||
<h2id="pipewire">pipewire(-pulse)</h2>
|
||||
</footer>
|
||||
</main>
|
||||
<?sh./footer.sh?>
|
||||
</body>
|
||||
</html>
|
||||
Loading…
Reference in New Issue