Long ago, when I was still a console newbie, I copied my friend’s bash configuration file. It had all the necessary stuff already included - aliases, colors, and most importantly: a nice prompt setup. I used it on all machines I had access to due to all the extra context it provided. For example, it would turn red when I was on a production machine, show me a current git branch from the repository I was working on, and indicate whether I had any changes to commit by showing a star.

Today there are a seemingly endless set of tools for configuring the command prompt to your liking, but back then things like Starship were completely non-existent. Most developers I know have customized their prompts in one way or another. This fact, combined with the feedback we’ve received from the Warp community, was good motivation to dig deeper into prompt customization in Warp.

In this post I’m going to tell you how we implemented support for PS1 in Warp, and why adding it was technically challenging. Along the way, we’ll also tackle the DCS (device control string) and some fun shell tricks!

Hey, Cieplik, what actually is PS1?

Glad you asked! PS1 is one of the few variables used by the shell to generate the prompt. As explained in the bash manual, PS1 represents the primary prompt string (hence the “PS”) - which is what you see most of the time before typing a new command in your terminal. On top of that there are actually few more - from PS0 to PS4, each of them executed and printed in different contexts of command execution. For example, you’ll see PS2 whenever the command has multiple lines as a secondary prompt string. And then zsh also provides RPS1, which displays the Prompt String on the right-hand side… there's a lot to work with.

Each of the Prompt String variables can easily be customized with the backslash-escaped special characters, but they can also contain shell function calls or even emojis, since most modern terminals support unicode.

With Warp in the works we made a decision to provide useful defaults for users to help them hit the ground running when using a new app. This included the prompt. We explicitly decided to not support PS1 in Warp, as it could clash with the bootstrapping script and PRECMD/PREEXEC hooks we use to create blocks of inputs and outputs. By implementing our own default prompt we had much more control over the user experience, and that was critical at the early stages. Eventually, however, it became obvious that our users really need their custom setups and lack of certain information on the prompt may actually slow them down. It came time for us to honor the user’s PS1 settings. Keep in mind that this is not our last word on this topic

What’s under the Prompt String’s hood?

In Warp, because we buffer the user’s input, we can’t allow the shell to directly render the prompt. Instead we use the metadata from the shell to do our own rendering. Our app receives it from the shell via the PRECMD hook, to do things like retrieve the current git branch, which is used in Warp’s default prompt. We use a special escape sequence and a JSON string packed with data to pass all the necessary information.

This is why we decided that the best way to support PS1 in Warp is to pass it as part of our PRECMD hook. The first challenge, however, comes from printing the variable into this JSON string.

Your PS1 is usually a set of variables, like information about the colors you want to see, but sometimes it can be a function call too. We have to translate it into a rendered Prompt String - a set of escape sequences and characters that tells the terminal the exact string to print and how to print it.

Here’s the example of an oh-my-zsh PS1 setting

> echo $PS1
%F{magenta}%n%f at %F{yellow}%m%f in %B%F{green}%~%f%b$(git_prompt_info)$(ruby_prompt_info)
$(virtualenv_info) $(prompt_char)

In zsh rendering a prompt is actually quite simple. You just print it. (and the shell expands all parameters for you via prompt parameter expansion.) We insert the output into our JSON and display it on our terminal.

Note that the middle prompt is the output of the print command, prepended with "rendered prompt" for clarity.

Bash does have a similar way of expanding prompt parameters as zsh does, so we could easily run echo ${PS1@P} in our script to get the rendered prompt and call it a day! Except…

That functionality has only been introduced in bash 4.4+. But Macs do not ship with that by default (this is related to a licensing change: since version 4.0, bash switched to a GPL license, which is not supported by Apple). The question, then, is how do we render the PS1 in older versions of bash without using any special tools or libraries?

The way we ended up solving this issue is by invoking a subshell, executing a very simple command in it, capturing the entire subshell output (including the prompt), and then manipulating it to only capture the prompt itself.

This is the final code we’re currently using in Warp to get the value of the rendered bash prompt:

$(echo -e "\n"  | PS1="$WARP_PS1" bash --norc -i 2>&1 | head -2 | tail -1)

Let me break that down, pipe by pipe:

echo -e "\n" prints an empty line in our subshell;
PS1="WARP_PS1" bash --norc -i 2>&1 we pass a previously captured $WARP_PS1 value as the PS1 in the subshell, and specify that no configuration files should be loaded (norc argument) to improve the performance of this operation; the -i flag denotes an interactive shell, while 2>&1 redirects the stderr to stdout, which allows us to capture the rendered prompt;
Head & tail operations simply help us manipulate the output to extract the prompt only.

That’s it. We've got our rendered prompt, we can start showing it in Warp!

Escaping the Prompt

Some things and places are really hard to escape, like the Russian city of Omsk, where even stray dogs cannot leave. Escaping in the terminal realm is really close to that experience, and Prompt String seems to make it even harder.

When juggling shell data and passing it to Warp via our PRECMD hook, we escape escape sequences that may break our JSON string with our magic sed invocation:

sed -E 's/(["\])/\\1/g; s/''\t''/\t/g; s/''\r''/\r/g; /\n/'

This is actually a series of replacements:

All double-quotes and backslashes are replaced with their escaped versions (\” and \);
\b (backspace), \t (tab), \f (form feed), \r (carriage return) are all replaced with their escaped versions;
We add an escaped \n (new line) on every line explicitly, since sed analyzes the input line-by-line and thus is not aware of the actual new lines.

This worked great, yet with the Prompt String you actually get many more things to escape: it’s not uncommon for the rendered prompt to include some extra bells and whistles (I mean the actual bell character - \a) and lots of other non-printable characters and escape sequences (most significant being \x1b aka \e or \033, which is literally ESC). Note that those sequences also had to be double escaped, to not only create a valid JSON string, but also not break our Rust implementation when unescaped! At least we were able to use the same tool for both supported shells this time!

In the first iteration, the final sed call looked a lot like:

sed -E 's/'$'\e''/\\e/g; s/'$'\a''/\\a/g;  s/(["\\])/\\\1/g; s/'$'\b''/\\b/g; s/'$'\t''/\\t/g; s/'$'\f''/\\f/g; s/'$'\r''/\\r/g; $!s/$/\\n/'

Back to the whiteboard

At this point, we started testing internally. It was then that we found a mysterious behavior.

A common tool used to customize the prompt in zsh - oh-my-zsh - is used by many team members. The prompt rendering worked for every theme...except the default oh-my-zsh theme. When the theme was used, the entire bootstrapping script would fail, leaving Warp in an unusable state.

Robby Russell’s oh-my-zsh prompt theme

It turned out the little arrow (➜) to the left was the root cause!

With shells and terminal emulators it always makes sense to analyze the actual bytes that are being processed. Let's check what’s under the hood of this little arrow:

This particular emoji consists of 4 bytes: e2, 9e, 9c and 0a. A careful reader may notice that both 9e and 9c bytes come from the extended ASCII table, but do they carry any special meaning? When in doubt, it’s always good to go back to the source - in this case, check the shell parser documentation. For completeness, let's quickly unpack what VT100 (and other VTs) is, and what this mysterious DCS is that are both mentioned in the linked docs:

VT100 was a Video Terminal introduced in the 70s. It was one of the first machines that allowed for cursor control with ANSI-escape codes, and added a bunch of other control options. Later on its spec became a de facto standard, and modern terminal emulator programs try to follow it (including Warp).

Control characters and control sequences in the terminal world are special ANSI-escape characters that control the terminal’s behavior. It can be anything from the cursor position, mouse control, even colors. DCS (device control string) is a special control sequence that is followed by a data string. You can find out more here.

unhook

When a device control string is terminated by ST, CAN, SUB or ESC, this action calls the previously selected handler function with an “end of data” parameter. This allows the handler to finish neatly.

As it turned out - the 9c byte carried a special meaning in a terminal world - it’s an ST (string terminator) escape sequence, which also happens to denote the unhook operation. As a result, the emoji in our JSON string would prematurely end the PRECMD hook, making it impossible for Warp to fully start.

From there the solution was simple - rather than trying to escape emojis and unicode characters that may have similar issues in the future, we simply encoded the entire Prompt String as a hexadecimal string. This completely eliminated the need for using sed and simplified our shell script.

Below you'll find a snippet with our current precmd function (example in bash):

   warp_escape_ps1 () {
           tr '\n\n' ' ' <<< "$*" | xxd -p | tr -d '\n'
        }

                # Format a string value according to JSON syntax - Adapted from https://github.com/lework/script.
        warp_escape_json () {
            # Explanation of the sed replacements (each command is separated by a `;`):
            # s/(["\\])/\\\1/g - Replace all double-quote (") and backslash (\) characters with the escaped versions (\" and \\)
            # s/\b/\\b/g - Replace all backspace characters with \b
            # s/\t/\\t/g - Replace all tab characters with \t
            # s/\f/\\f/g - Replace all form feed characters with \f
            # s/\r/\\r/g - Replace all carriage return characters with \r
            # $!s/$/\\n/ - On every line except the last, insert the \n escape at the end of the line
            #              Note: sed acts line-by-line, so it doesn't see the literal newline characters to replace
            #
            # tr -d '\n' - Remove the literal newlines from the final output
            #
            # Additional note: In a shell script between single quotes ('), no escape sequences are interpreted.
            # To work around that and insert the literal values into the regular expressions, we stop the single-quote,
            # then add the literal using ANSI-C syntax ($'\t'), then start a new single-quote. That is the meaning
            # behind the various `'$'\b''` blocks in the command. All of these separate strings are then concatenated
            # together to form the full argument to send to sed.
            sed -E 's/(["\\])/\\\1/g; s/'$'\b''/\\b/g; s/'$'\t''/\\t/g; s/'$'\f''/\\f/g; s/'$'\r''/\\r/g; $!s/$/\\n/' <<<"$*" | tr -d '\n'
        }

        warp_precmd () {
            # $? is relative to the process so we MUST check this first
            # or else the exit code will correspond to the commands
            # executed within this block instead of the actual last
            # command that was run.
            local exit_code=$?
            # Clear the prompt again before the command is rendered as it could
            # have been reset by the user's bashrc or by setting the variable
            # on the command line.
            if [[ -n $PS1 ]]; then
              WARP_PS1="$PS1"
            fi
            unset PS1
            unset PROMPT
            # Escaped PS1 variable
            local escaped_ps1
            if [[ $WARP_FEATURE_FLAG_HONOR_PS1 == "1" ]]; then
              # Tricking the shell into rendering the prompt
              # Note that in more modern versions of bash we could use ${PS1@P} to achieve the same,
              # but macOs comes by default with a much older version of bash, and we want to be compatible.
              deref_ps1=$(echo -e "\n"  | PS1="$WARP_PS1" bash --norc -i 2>&1 | head -2 | tail -1)
              escaped_ps1=$(warp_escape_ps1 "$(echo "$deref_ps1")")
            fi
            # Flush history
            history -a
            # Reset the custom kill-whole-line binding as the user's bashrc (which is sourced after bashrc_warp)
            # could have added another bind. This won't have any user-impact because these shortcuts are only run
            # in the context of the bash editor, which isn't displayed in Warp.
            bind -r '"\C-p"'
            bind "\C-p":kill-whole-line
            local escaped_pwd
            escaped_pwd=$(warp_escape_json "$PWD")
            local escaped_virtual_env=""
            if [ ! -z "$VIRTUAL_ENV" ]; then
                escaped_virtual_env=$(warp_escape_json "$VIRTUAL_ENV")
            fi
            local escaped_conda_env=""
            if [ ! -z "$CONDA_DEFAULT_ENV" ]; then
                escaped_conda_env=$(warp_escape_json "$CONDA_DEFAULT_ENV")
            fi
            local git_branch
            git_branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "")
            local escaped_git_branch
            escaped_git_branch=$(warp_escape_json "$git_branch")
            # At this point, escaped prompt looks something like
            # \\u{001B}\\u{005B}\\u{0030}\\u{0031}\\u{003B} ...
            # We need to maintain the double quoting of \\u in the message that
            # is sent otherwise the receiving side will interpret the value
            # as JS string literals of the form \uHEX, and will include
            # ctrl characters (like ESC) in the json, which will cause a JSON
            # parse error.
            # Note WARP_SESSION_ID doesn't need to be escaped since it's a number
            local escaped_json="{\"hook\": \"Precmd\", \"value\": {\"pwd\": \"$escaped_pwd\", \"ps1\": \"$escaped_ps1\", \"git_branch\": \"$escaped_git_branch\", \"virtual_env\": \"$escaped_virtual_env\", \"conda_env\": \"$escaped_conda_env\", \"exit_code\": $exit_code, \"session_id\": $WARP_SESSION_ID}}"
            warp_send_message "$escaped_json"
        }

Future of the prompt

Warp now has the option to honor the user's PS1 setting. It works with most configurations, though you’ll find the full compatibility table in our docs.

It is not, however, the last time you'll hear from us about working on the prompt. Our key product principles include fixing the UI and providing a great out-of-the box experience. One of the ideas we're currently exploring is Context Chips - our low-calorie approach to snacking on bite-size information.

We’re working on plenty of interesting problems just like this one at Warp; and we’re hiring! If you want to be a part of our team and work together on improving the day-to-day life of developers from around the world - check out our careers page.

What’s so special about PS1? Fun with customizing Bash command prompts

Agata Cieplik