Integrated_environment

Comparison of command shells

Comparison of command shells

Add article description


A command shell is a command-line interface to interact with and manipulate a computer's operating system.

Bash, the default shell on many GNU/Linux systems.

General characteristics

More information Shell, Usual environment ...

Interactive features

More information Shell, Commandnamecompletion ...

Background execution

Background execution allows a shell to run a command without user interaction in the terminal, freeing the command line for additional work with the shell. POSIX shells and other Unix shells allow background execution by using the & character at the end of command. In PowerShell, the Start-Process[43] or Start-Job[44] cmdlets can be used.

Completions

Command-line completion in Bash.

Completion features assist the user in typing commands at the command line, by looking for and suggesting matching words for incomplete ones. Completion is generally requested by pressing the completion key (often the Tab ↹ key).

Command name completion is the completion of the name of a command. In most shells, a command can be a program in the command path (usually $PATH), a builtin command, a function or alias.

Path completion is the completion of the path to a file, relative or absolute.

Wildcard completion is a generalization of path completion, where an expression matches any number of files, using any supported syntax for file matching.

Variable completion is the completion of the name of a variable name (environment variable or shell variable). Bash, zsh, and fish have completion for all variable names. PowerShell has completions for environment variable names, shell variable names and — from within user-defined functions — parameter names.

Command argument completion is the completion of a specific command's arguments. There are two types of arguments, named and positional: Named arguments, often called options, are identified by their name or letter preceding a value, whereas positional arguments consist only of the value. Some shells allow completion of argument names, but few support completing values.

Bash, zsh and fish offer parameter name completion through a definition external to the command, distributed in a separate completion definition file. For command parameter name/value completions, these shells assume path/filename completion if no completion is defined for the command. Completion can be set up to suggest completions by calling a shell function.[45] The fish shell additionally supports parsing of man pages to extract parameter information that can be used to improve completions/suggestions. In PowerShell, all types of commands (cmdlets, functions, script files) inherently expose data about the names, types and valid value ranges/lists for each argument. This metadata is used by PowerShell to automatically support argument name and value completion for built-in commands/functions, user-defined commands/functions as well as for script files. Individual cmdlets can also define dynamic completion of argument values where the completion values are computed dynamically on the running system.

Command history

Users of a shell may find themselves typing something similar to what they have typed before. Support for command history means that a user can recall a previous command into the command-line editor and edit it before issuing the potentially modified command.

Shells that support completion may also be able to directly complete the command from the command history given a partial/initial part of the previous command.

Most modern shells support command history. Shells which support command history in general also support completion from history rather than just recalling commands from the history. In addition to the plain command text, PowerShell also records execution start- and end time and execution status in the command history.

Mandatory argument prompt

Mandatory arguments/parameters are arguments/parameters which must be assigned a value upon invocation of the command, function or script file. A shell that can determine ahead of invocation that there are missing mandatory values, can assist the interactive user by prompting for those values instead of letting the command fail. Having the shell prompt for missing values will allow the author of a script, command or function to mark a parameter as mandatory instead of creating script code to either prompt for the missing values (after determining that it is being run interactively) or fail with a message.

PowerShell allows commands, functions and scripts to define arguments/parameters as mandatory. The shell determines prior to invocation if there is any mandatory arguments/parameters which have not been bound, and will then prompt the user for the value(s) before actual invocation. [46]

Automatic suggestions

Command-line completion in PowerShell.

Shells featuring automatic suggestions display optional command-line completions as the user types. The PowerShell and fish shells natively support this feature; pressing the Tab ↹ key inserts the completion.

Implementations of this feature can differ between shells; for example, PowerShell[47] and zsh[48] use an external module to provide completions, and fish derives its completions from the user's command history.[49]

Directory history, stack or similar features

Shells may record a history of directories the user has been in and allow for fast switching to any recorded location. This is referred to as a "directory stack". The concept had been realized as early as 1978[50] in the release of the C shell (csh).

PowerShell allows multiple named stacks to be used. Locations (directories) can be pushed onto/popped from the current stack or a named stack. Any stack can become the current (default) stack. Unlike most other shells, PowerShell's location concept allow location stacks to hold file system locations as well as other location types like e.g. Active Directory organizational units/groups, SQL Server databases/tables/objects, Internet Information Server applications/sites/virtual directories.

Command line interpreters 4DOS and its graphical successor Take Command Console also feature a directory stack.

Implicit directory change

A directory name can be used directly as a command which implicitly changes the current location to the directory.

This must be distinguished from an unrelated load drive feature supported by Concurrent DOS, Multiuser DOS, System Manager and REAL/32, where the drive letter L: will be implicitly updated to point to the load path of a loaded application, thereby allowing applications to refer to files residing in their load directory under a standardized drive letter instead of under an absolute path.[51]

Autocorrection

Zsh autocompletion and autocorrection demo for a telnet program.

When a command line does not match a command or arguments directly, spell checking can automatically correct common typing mistakes (such as case sensitivity, missing letters). There are two approaches to this; the shell can either suggest probable corrections upon command invocation, or this can happen earlier as part of a completion or autosuggestion.

The tcsh and zsh shells feature optional spell checking/correction, upon command invocation.

Fish does the autocorrection upon completion and autosuggestion. The feature is therefore not in the way when typing out the whole command and pressing enter, whereas extensive use of the tab and right-arrow keys makes the shell mostly case insensitive.

The PSReadLine[31] PowerShell module (which is shipped with version 5.0) provides the option to specify a CommandValidationHandler ScriptBlock which runs before submitting the command. This allows for custom correcting of commonly mistyped commands, and verification before actually running the command.

Progress indicator

A shell script (or job) can report progress of long running tasks to the interactive user.

Unix/Linux systems may offer other tools support using progress indicators from scripts or as standalone-commands, such as the program "pv".[52] These are not integrated features of the shells, however.

PowerShell has a built-in command and API functions (to be used when authoring commands) for writing/updating a progress bar. Progress bar messages are sent separates from regular command output and the progress bar is always displayed at the ultimate interactive users console regardless of whether the progress messages originates from an interactive script, from a background job or from a remote session.

Colored directory listings

JP Software command-line processors provide user-configurable colorization of file and directory names in directory listings based on their file extension and/or attributes through an optionally defined %COLORDIR% environment variable.

For the Unix/Linux shells, this is a feature of the ls command and the terminal.

Text highlighting

The command line processors in DOS Plus, Multiuser DOS, REAL/32 and in all versions of DR-DOS support a number of optional environment variables to define escape sequences allowing to control text highlighting, reversion or colorization for display or print purposes in commands like TYPE. All mentioned command line processors support %$ON% and %$OFF%. If defined, these sequences will be emitted before and after filenames. A typical sequence for %$ON% would be \033[1m in conjunction with ANSI.SYS, \033p for an ASCII terminal or \016 for an IBM or ESC/P printer. Likewise, typical sequences for %$OFF% would be \033[0m, \033q, \024, respectively. The variables %$HEADER% and %$FOOTER% are only supported by COMMAND.COM in DR-DOS 7.02 and higher to define sequences emitted before and after text blocks in order to control text highlighting, pagination or other formatting options.

For the Unix/Linux shells, this is a feature of the terminal.

Syntax highlighting

A defining feature of the fish shell is built-in syntax highlighting, As the user types, text is colored to represent whether the input is a valid command or not (the executable exists and the user has permissions to run it), and valid file paths are underlined.[53]

An independent project offers syntax highlighting as an add-on to the Z Shell (zsh).[54] This is not part of the shell, however.

PowerShell provides customizable syntax highlighting on the command line through the PSReadLine[31] module. This module can be used with PowerShell v3.0+, and is bundled with v5.0 onwards. It is loaded by default in the command line host "powershell.exe" since v5.0.[55]

Take Command Console (TCC) offers syntax highlighting in the integrated environment.

Context sensitive help

4DOS, 4OS2, 4NT / Take Command Console and PowerShell (in PowerShell ISE) looks up context-sensitive help information when F1 is pressed.

Zsh provides various forms of configurable context-sensitive help as part of its run-help widget, _complete_help command, or in the completion of options for some commands.

The fish shell provides brief descriptions of a command's flags during tab completion.


Programming features

More information Shell, Functions ...

String processing and filename matching

More information Shell, String processing ...

Inter-process communication

More information Shell, Pipes ...

Keystroke stacking

In anticipation of what a given running application may accept as keyboard input, the user of the shell instructs the shell to generate a sequence of simulated keystrokes, which the application will interpret as a keyboard input from an interactive user. By sending keystroke sequences the user may be able to direct the application to perform actions that would be impossible to achieve through input redirection or would otherwise require an interactive user. For example, if an application acts on keystrokes, which cannot be redirected, distinguishes between normal and extended keys, flushes the queue before accepting new input on startup or under certain conditions, or because it does not read through standard input at all. Keystroke stacking typically also provides means to control the timing of simulated keys being sent or to delay new keys until the queue was flushed etc. It also allows to simulate keys which are not present on a keyboard (because the corresponding keys do not physically exist or because a different keyboard layout is being used) and therefore would be impossible to type by a user.

Security features

More information Shell, Secure (password) prompt ...

Secure prompt

Some shell scripts need to query the user for sensitive information such as passwords, private digital keys, PIN codes or other confidential information. Sensitive input should not be echoed back to the screen/input device where it could be gleaned by unauthorized persons. Plaintext memory representation of sensitive information should also be avoided as it could allow the information to be compromised, e.g., through swap files, core dumps etc.[70]

The shells bash, zsh and PowerShell offer this as a specific feature.[71][72] Shells which do not offer this as a specific feature may still be able to turn off echoing through some other means. Shells executing on a Unix/Linux operating system can use the stty external command to switch off/on echoing of input characters.[73] In addition to not echoing back the characters, PowerShell's -AsSecureString option also encrypts the input character-by-character during the input process, ensuring that the string is never represented unencrypted in memory where it could be compromised through memory dumps, scanning, transcription etc.

Execute permission

Some operating systems define an execute permission which can be granted to users/groups for a file when the file system itself supports it.

On Unix systems, the execute permission controls access to invoking the file as a program, and applies both to executables and scripts. As the permission is enforced in the program loader, no obligation is needed from the invoking program, nor the invoked program, in enforcing the execute permission  this also goes for shells and other interpreter programs. The behaviour is mandated by the POSIX C library that is used for interfacing with the kernel. POSIX specifies that the exec family of functions shall fail with EACCESS (permission denied) if the file denies execution permission (see execve  System Interfaces Reference, The Single UNIX Specification, Version 4 from The Open Group).

The execute permission only applies when the script is run directly. If a script is invoked as an argument to the interpreting shell, it will be executed regardless of whether the user holds the execute permission for that script.

Although Windows also specifies an execute permission, none of the Windows-specific shells block script execution if the permission has not been granted.

Restricted shell subset

Several shells can be started or be configured to start in a mode where only a limited set of commands and actions is available to the user. While not a security boundary (the command accessing a resource is blocked rather than the resource) this is nevertheless typically used to restrict users' actions before logging in.

A restricted mode is part of the POSIX specification for shells, and most of the Linux/Unix shells support such a mode where several of the built-in commands are disabled and only external commands from a certain directory can be invoked.[74][75]

PowerShell supports restricted modes through session configuration files or session configurations. A session configuration file can define visible (available) cmdlets, aliases, functions, path providers and more.[76]

Safe data subset

Scripts that invoke other scripts can be a security risk as they can potentially execute foreign code in the context of the user who launched the initial script. Scripts will usually be designed to exclusively include scripts from known safe locations; but in some instances, e.g. when offering the user a way to configure the environment or loading localized messages, the script may need to include other scripts/files.[77] One way to address this risk is for the shell to offer a safe subset of commands which can be executed by an included script.

PowerShell data sections can contain constants and expressions using a restricted subset of operators and commands.[78] PowerShell data sections are used when e.g. localized strings needs to be read from an external source while protecting against unwanted side effects.

Notes

  1. Since mid 1990s.
  2. if compiled with -DACCT.
  3. MS-DOS and Windows component – covered by a valid license for MS-DOS or Microsoft Windows.
  4. OS/2 component – covered by a valid license for OS/2.
  5. Command extensions enabled, or "CMD /X".
  6. Windows component – covered by a valid license for Microsoft Windows.
  7. Microsoft PowerShell is installed by default on Windows 7 and later. It is an optional download for users of Windows Vista or Windows XP.
  8. current versions from Jörg Schilling.
  9. Alt-Shift-8 or Alt-* will expand to the full matching list of filenames.
  10. Available through the DOSKEY add-on.
  11. Available in DR-DOS through HISTORY.
  12. Alternatively available through the DOSKEY add-on as well.
  13. Alternatively available in DR-DOS through HISTORY as well.
  14. TCC has special prompt functions for Yes, No, Cancel, Close, Retry.
  15. Handled by rio, GNU readline, editline or vrl.
  16. The fish shell is an interactive character based input/output surface.
  17. Not available as a shell built-in. External FINDSTR /R command is available in most Windows releases.
  18. PowerShell leverages the full .NET regular expression engine which features named captures, zero-width lookahead/-behind, greedy/non-greedy, character classes, level counting etc.
  19. xautomation and xdotool can be used to generate keystrokes under X Window System; or a program can be run in a pseudoterminal to be able to control it (as with the expect tool).
  20. The shell can use the stty utility to suppress echoing of typed characters to the screen. This requires multiple steps: 1. reading the current echo state, 2. switching echo off, 3. reading the input, 4. switching echo state back to the original state.
  21. The execute permission is enforced by a separate program, the program loader, by refusing to invoke the interpreter (possibly a shell) specified by the script's hashbang. The interpreter does not enforce the execute permission if invoked directly as the program loader would, with the file as an argument; this only requires read permission, as does piping the file as input to the interpreter, in which case the interpreter cannot see the execute permission.
  22. The zsh and fish shells also honor the execute permission for command completion.
  23. Under DR-DOS the password separator for file and directory passwords is a semicolon. This is also supported under 4DOS for as long as the command does not support include lists. Under 4DOS, the password separator must be doubled for all commands supporting include lists in order to distinguish passwords from include lists. Commands not supporting include lists accept both forms. DR-DOS 7.02 and higher optionally accept a doubled semicolon as well, so that doubled semicolons work under both COMMAND.COM and 4DOS regardless of the command executed.
  24. DR-DOS supports file passwords for read/write/delete and optionally execute permissions. Files are not protected by default, but the system can be set up so that f.e. batch scripts require a password to read.
  25. INPUT /P and INKEY /P echoes back asterisks for each typed character.
  26. Read-Host -AsSecureString reads a string of characters from the input device into an encrypted string, one character at a time thus ensuring that there is no memory image of the clear text which could be gleaned from scanning memory, or from crash dumps, memory dumps, paging files, log files or similar.
  27. PowerShell script files (.ps1 files) are by default associated with the Notepad editor, not with the PowerShell execution engine. Invoking a .ps1 file will launch Notepad rather than executing the script.
  28. Startup scripts per computer/user can import modules and expose a subset the commands/functions available in the modules.

References

  1. A platform independent version based on the historical UNIX V7 original source code is available from Geoff Collyer
  2. The historic UNIX V7 version is available under a BSD-style license through The Unix Heritage Society and others.
  3. A platform independent version based on the SVr4/Solaris source code is available from Jörg Schilling
  4. Ferrell, John, "Chapter 2. Default Shell", FreeBSD Quickstart Guide for Linux Users, The FreeBSD Documentation Project, retrieved 2015-07-24
  5. IEEE and The Open Group (2008). IEEE 1003.1 Standard for Information Technology – Portable Operating System Interface (POSIX): Shell and Utilities, Issue 7.
  6. As part of IEEE Std.1003.2-1992 (POSIX.2); integrated into IEEE Std.1003.1 with the 2001 revision.
  7. Fox, Brian (1989-06-07). Tower Jr., Leonard H. (ed.). "Bash is in beta release!". Newsgroup: gnu.announce. Usenet: [email protected]. Retrieved 2010-10-28.
  8. Cooper, Mendel, "Chapter 37.3.2. Bash, version 4.2", Advanced Bash-Scripting Guide, The Linux Documentation Project, retrieved 2015-04-30, "Bash now supports the \u and \U Unicode escape."
  9. "FreeBSD Quickstart Guide for Linux® Users". FreeBSD Documentation Portal. Retrieved 2024-02-04.
  10. Sussman, Ann (1988-12-26). "Hamilton C Shell Speeds Development Of OS/2 Applications" (PDF). PC Week (1988-12-26 - 1989-01-02): 37. Retrieved 2010-11-22.
  11. Gomes, Ron (1983-06-09). "Toronto USENIX Conference Schedule (tentative)". Newsgroup: net.usenix. Retrieved 2010-12-29.
  12. Harris, Guy (1983-10-10). "csh question". Newsgroup: net.flame. Retrieved 2010-12-29.
  13. The zsh command line editor is fully configurable and can allow mouse support in various ways such as with Stéphane Chazelas's mouse.zsh.
  14. Lefevre, Vincent (2015-02-11). "multi-digit file descriptors". zsh-users (Mailing list). Retrieved 2021-12-23.
  15. "HP OpenVMS DCL Dictionary". Archived from the original on 2007-03-25. Retrieved 2009-03-23.
  16. Larabel, Michael (2021-03-23). "Plan 9 Copyright Transferred To Foundation, MIT Licensed Code Released". Phoronix. Retrieved 2021-03-28.
  17. Liljencrantz, Axel (2005-05-17). "Fish - The friendly interactive shell". Retrieved 2013-04-08.
  18. Soller, Jeremy (2015-11-15). "d79c8f511573fb7710abc63b4236a40022914520". Retrieved 2019-08-03.
  19. "[Z Shell] Completion System". Zsh.sourceforge.io. Retrieved 2015-02-24.
  20. This applies only on reserved words and other syntactic features.
  21. e.g. via 3rd party such as zsh-autosuggestions
  22. zsh does not feature syntax highlighting, but a 3rd party project exists which offers this capability as an add-on: zsh-syntax-highlighting
  23. Paul, Matthias R. (1997-10-02) [1997-09-29]. "Caldera OpenDOS 7.01/7.02 Update Alpha 3 IBMBIO.COM - README.TXT and BOOT.TXT - A short description of how OpenDOS is booted". Archived from the original on 2003-10-04. Retrieved 2009-03-29.
  24. "Windows PowerShell 5.0". Archived from the original on 17 September 2016. Retrieved 8 September 2016.
  25. "Windows PowerShell Integrated Scripting Environment (ISE)". Microsoft Technet. Retrieved 2015-09-12.
  26. sdwheeler. "Write-Host (Microsoft.PowerShell.Utility) - PowerShell". docs.microsoft.com. Retrieved 2022-01-18.
  27. Push-Location (with alias pushd) and Pop-Location (with alias popd) allows multiple location types (directories of file systems, organizational units of Active Directory, nodes of Windows Registry etc) to be pushed onto and popped from location stacks.
  28. The $host.ui.PromptForChoice function allows for a menu-style prompt for choices. The prompt works from background jobs as well as from remote sessions, displaying the menu prompt on the console of the controlling session.
  29. The Write-Progress cmdlet writes a progress bar which can indicate percentage, remaining seconds etc. The progress bar messages work from background jobs or remote sessions in addition to interactive scripts, i.e. the progress bar is displayed on the console of the controlling session, not as part of the regular output.
  30. The Show-Command cmdlet inspects the command definition and opens an interactive windows with a named input field for each parameter/switch
  31. "fish: Documentation". Section Tab completion. Retrieved 2016-01-10.
  32. "set_color - set the terminal color — fish-shell 3.1.2 documentation". fishshell.com. Archived from the original on 2020-02-17. Retrieved 2021-02-23.
  33. sdwheeler. "Start-Process (Microsoft.PowerShell.Management) - PowerShell". learn.microsoft.com. Retrieved 2022-10-21.
  34. sdwheeler. "Start-Job (Microsoft.PowerShell.Core) - PowerShell". learn.microsoft.com. Retrieved 2022-10-21.
  35. "zsh: 20. Completion System". Zsh.sourceforge.io. 2013-03-06. Retrieved 2013-08-18.
  36. "Use PowerShell to Make Mandatory Parameters". Blogs.technet.com. 2011-05-22. Retrieved 2015-02-24.
  37. sdwheeler. "What's New in the PowerShell 5.0 ISE - PowerShell". docs.microsoft.com. Retrieved 2021-07-25.
  38. "Interactive use — fish-shell 3.3.1 documentation". fishshell.com. Retrieved 2021-07-25.
  39. Hahn, Harley (2009). Harley Hahn's guide to Unix and Linux. Boston: McGraw-Hill Higher Education. ISBN 978-0-07-313361-4. OCLC 184828059.
  40. "fish: Tutorial". fishshell.com. Retrieved 2022-10-21.
  41. sdwheeler. "PSReadLine Module - PowerShell". learn.microsoft.com. Retrieved 2023-04-26.
  42. "Ash Variants". Archived from the original on 2010-03-10. Retrieved 2014-12-15.
  43. "Set - Environment Variable - Windows CMD". SS64.com. Retrieved 2015-02-24.
  44. "How to loop through array in batch?". Stack Overflow. Retrieved 2015-02-24.
  45. The .NET System.Math class defines mathematical functions that can be used through the shortcut [Math], e.g. [Math]::Sin for the sinus function.
  46. "Get closure with GetNewClosure". devblogs.microsoft.com. 2009-03-27. Retrieved 2022-09-12.
  47. Hamilton C shell Language reference: Wildcarding and pattern matching, Hamilton Laboratories, retrieved 2013-10-29, ... Indefinite Directory: match any number of directory levels – zero or more – whatever it takes to make the rest of the pattern match.
  48. Seebach, Peter (2008-11-21). Beginning Portable Shell Scripting: From Novice to Professional. Expert's voice in open source. Apress (published 2008). p. 149. ISBN 9781430210436. Retrieved 2014-09-17. Brace expansion is available in ksh93, pdksh, bash, and zsh.
  49. Zsh offers a variety of globbing options.
  50. sdwheeler. "about Operators - PowerShell". docs.microsoft.com. Retrieved 2022-01-18.
  51. Brothers, Hardin; Rawson, Tom; Conn, Rex C.; Paul, Matthias R.; Dye, Charles E.; Georgiev, Luchezar I. (2002-02-27). 4DOS 8.00 online help.
  52. "About Data Sections". Technet.microsoft.com. Retrieved 2015-02-24.
  53. "Ubuntu Manpage: rc - shell". Manpages.ubuntu.com. 2003-07-17. Retrieved 2015-02-24.
  54. Provos, Niels. "Encrypting Virtual Memory". Center for Information Technology Integration, University of Michigan. Retrieved 2012-12-20.
  55. "bash - GNU Bourne-Again SHell". read -s Silent mode. If input is coming from a terminal, characters are not echoed.
  56. "Using the Read-Host Cmdlet". By adding the -assecurestring parameter you can mask the data entered at the prompt
  57. "Linux / Unix Command: stty". Linux.about.com. Archived from the original on 2015-02-25. Retrieved 2015-02-24.
  58. "man sh - shell, the standard command language interpreter / posix" (in French). Pwet.fr. Archived from the original on 2014-12-21. Retrieved 2013-08-18.
  59. "Bash Reference Manual: The Restricted Shell". Gnu.org. 2010-12-28. Retrieved 2013-08-18.
  60. "New-PSSessionConfigurationFile". Technet.microsoft.com. Retrieved 2013-08-18.
  61. Albing, Carl; Vossen, J. P.; Newham, Cameron (2007). Bash cookbook (1st ed.). Sebastopol, California, USA: O'Reilly Media. ISBN 978-0-596-52678-8. [...] is hardly what one thinks of as a passive list of configured variables. It can run other commands (e.g., cat) and use if statements to vary its choices. It even ends by echoing a message. Be careful when you source something, as it's a wide open door into your script.
  62. "About Data Sections". Microsoft. Retrieved 2012-12-18.

Share this article:

This article uses material from the Wikipedia article Integrated_environment, and is written by contributors. Text is available under a CC BY-SA 4.0 International License; additional terms may apply. Images, videos and audio are available under their respective licenses.