Shell Tools and Scripting

Shell Scripting

• foo=bar differ from foo = bar

foo=bar
echo "$foo"
# prints bar
echo '$foo'
# prints $foo

• if, case, while and for & function

mcd () {
  mkdir -p "$1"
  cd "$1"
}

Special Characters

More info: Chapter 3. Special Characters

• $0 - Name of the script
• $1 to $9 - Arguments to the script. $1 is the first argument and so on.
• $@ - All the arguments
• $# - Number of arguments
• $? - Return code of the previous command
• $$ - Process identification number (PID) for the current script
• !! - Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re-execute the command with sudo by doing sudo !!
• $_ - Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typing Esc followed by .

exit codes and short-circuiting operators

• && (and operator) and || (or operator)

• $( CMD ), e.g: for file in $(ls)

• <( CMD ), e.g: diff <(ls foo) <(ls bar)

• What is the difference between test, [ and [[ ?

shell globbing （通配符匹配）

• Wildcards: ? & *
• Curly braces: {}, expand, e.g: convert image.{png,jpg} => convert image.png image.jpg

convert image.{png,jpg}
# Will expand to
convert image.png image.jpg

cp /path/to/project/{foo,bar,baz}.sh /newpath
# Will expand to
cp /path/to/project/foo.sh /path/to/project/bar.sh /path/to/project/baz.sh /newpath

# Globbing techniques can also be combined
mv *{.py,.sh} folder
# Will move all *.py and *.sh files


mkdir foo bar
# This creates files foo/a, foo/b, ... foo/h, bar/a, bar/b, ... bar/h
touch {foo,bar}/{a..h}
touch foo/x bar/y
# Show differences between files in foo and bar
diff <(ls foo) <(ls bar)
# Outputs
# < x
# ---
# > y

• #!/PATH/TO/PROGRAME/ - "shebang line(Unix)"

  • the env command

• differences between shell functions and scripts

• using export

• Shell Tools

  • shellcheck - A shell script static analysis tool

  • Finding how to use commands

    • man
    • For interactive tools such as the ones based on ncurses, help for the commands can often be accessed within the program using the :help command or typing ?.
    • TLDR pages

  • Finding files

    • find or => fd (simple, fast, and user-friendly alternative, in rust)

    •   # Find all directories named src
        find . -name src -type d
        # Find all python files that have a folder named test in their path
        find . -path '*/test/*.py' -type f
        # Find all files modified in the last day
        find . -mtime -1
        # Find all zip files with size in range 500k to 10M
        find . -size +500k -size -10M -name '*.tar.gz'
      

    • # Delete all files with .tmp extension
      find . -name '*.tmp' -exec rm {} \;
      # Find all PNG files and convert them to JPG
      find . -name '*.png' -exec convert {} {}.jpg \;
      
      

    • locate , ref: locate vs find: usage, pros and cons of each other

  • Finding code

    • grep - matching patterns from the input text

      • -C for getting Context around the matching line
      • -v for inverting the match
      • Alternative: ack, ag and rg.

            # Find all python files where I used the requests library
            rg -t py 'import requests'
            # Find all files (including hidden files) without a shebang line
            rg -u --files-without-match "^#!"
            # Find all matches of foo and print the following 5 lines
            rg foo -A 5
            # Print statistics of matches (# of matched lines and files )
            rg --stats PATTERN

• Finding shell commands
  - history
  - Ctrl+R, perform backwards search through your history
  - This can also be enabled with the UP/DOWN arrows in zsh.
  - A nice addition on top of Ctrl+R comes with using fzf bindings.
  - the fish shell
  - HISTCONTROL=ignorespace to your .bashrc or setopt HIST_IGNORE_SPACE to your .zshrc.
  - If you make the mistake of not adding the leading space, you can always manually remove the entry by editing your .bash_history or .zhistory.

  • Directory Navigation

    • writing shell aliases or creating symlinks with ln -s
    • fasd and autojump.
      • GitHub - ajeetdsouza/zoxide: A smarter cd command. Supports all major shells.
    • get an overview of a directory structure: tree, broot

  • xargs command which will execute a command using STDIN as arguments

  • using brew.

array
an associative array

BashFAQ - Greg's Wiki