Handle Long Output Like A Pro
If you are working at the Linux shell, then every now and then it will happen that the output a command-line gives you is more extensive than the amount of data your terminal-window can display.
If this happens, the output not fitting into the terminal anymore simply scrolls out at the top and is not visible anymore.
But what if this is the output you’re interested in?
Of course you can have a look at the scrollbars your terminal window may give you. (perhaps you’ve opened a graphical terminal window or you are connected via putty)
Then take the mouse and simply scroll up.
But wait! What if the terminal doesn’t give you a scrollbar?
What if you are accessing the Linux system from the “real” console, for instance within your virtualization environment VMware vSphere or Hyper-V?
Aaand … don’t the cool guys always try hard to avoid using the mouse? ;-)
Well - because I know you are one of these cool guys, I have three ways for you to handle massive text-output at the command-line
Useful key-combinations to scroll through the terminal
Every terminal you are working on gives you two key combinations to scroll through past pages of the terminal:
<shift> + <PageUp> and <shift> + <PageDown>
Every time you hold down the shift-key and press “Page-Up”, the terminal will scroll up one page. You can repeat this until the end of the output-cache of the current terminal.
If you use <shift> + <PageDown> instead - you can scroll page-after-page back until you reach the active output page again.
The output-pages you can scroll through aren’t infinite - sometimes several hundreds and sometimes only a few. But you will always have access to the last pages that have just flown out of the terminal.
So instead of using the mouse - simply use these key combinations to scroll through the output.
This works in every terminal you may be working in: From the local console if you are sitting directly in front of a physical server, through the window of putty - if you are connected from your windows-pc to the Linux system - and also if you are working at a terminal of a graphical desktop-environment on a Linux system.
A second way to handle larger amounts of output is to use very helpful little tools
Let’s pretend we have the command “ls -l /etc” that generates a lot of output we are interested in.
The traditional pager “more”
The first tool that makes our life much more easy here is the tool “more”. To use it, we append a vertical bar “|” to the command line followed by this command:
ls -l /etc | more
(we “pipe” the output of “ls” to the command “more”, as the cool guys would call it)
Now we see only the first page of the huge output the ls command gives us.
To see the second page, simply press spacebar.
If you wanna scroll not a whole page but only a single line, you can use the enter key.
Because this command gives you page-after-page for the output, the command more is often called a “pager”.
You can repeatedly push the spacebar or enter, until you reach the end of the output.
If you have seen what you were looking for, you can leave the “more” command anytime you want by pressing “q” that stands for quit.
Often “less” is the better “more”
A main advantage of the “more” command is that it is available on any Linux system.
A disadvantage is, you cannot scroll back. Once you have scrolled over a page of output, you have to start the procedure from the very beginning.
A more comfortable way to scroll through text output is the command less:
ls -l /etc | less
The less command also starts by printing out only the first page.
And you can use it just like the “more” command by hitting space or enter.
But you don’t have to remember when to use space or enter: For navigation through the data you can here simply use the cursor-keys or PageDown/PageUp. And these keys let you also scroll backward!
Very comfortable, isn’t it?
To leave the less command, use the “q” key too.
Sometimes the less command isn’t available by default on a system. But give it a try, nothing to lose …
A third way would be to
Temporarily cache the data into a file
With this approach, you do not analyse the data you get in realtime, but you store them for later processing into a temporary file.
You do this by simply “redirecting” the command-line output. (that’s the official name for sending the output of a command-line somewhere else then to your terminal).
To redirect the output of a command to a file, simply append a “greater-than sign” (“>”) like this:
ls -l /etc > etc_list.txt
You can then later on process the data from this temporary file with other tools you know: Open them via your favorite editor like vi, nano or mcedit or transfer the data to your Windows desktop via WinSCP for instance.
Two important side-notes here:
First: Avoid destroying important data
If you are redirecting the output of the command line to a file, then the file you specify will always be created as a brand new file. No matter, if a file with the given name already exists or not.
So be careful with specifying a new uniq filename to avoid losing important data.
If you want to avoid losing data, you could also use a pair of “greater-than signs” like this:
ls -l /etc >> etc_list.txt
In this way, you are always appending the data to the file, should the file exist already.
Second: Take care of any additional data from stderr
The redirect with the “naked” greater-than sign only redirects the data stream called stdout.
This data stream has all the data your command “normally” prints out to the terminal.
What this data stream typically not shows you are the error messages a command line may give you. They are contained in a datastream called stderr.
This data is also displayed at your terminal but will not be catched by the “default” redirection.
If you want to redirect all the data as you see it at the terminal to a single file (including all the possible error messages) - append the additional pattern “2>&1” at the end of your command line like this:
find /etc -name "pass*" > list.txt 2>&1
Filter for the interesting lines only with “grep”
I give you the command “grep” here as a sort of ninja-way for handling the output.
If you are only interested in certain lines of the output and these lines contain a specific word you know, then use grep to print only the lines containing this word:
ls -l /etc | grep passwd
If you wanna go really fancy here, try to leverage the power of a regular expression here instead of only searching for a single word.
As a homework challenge: What is the following command line searching for?
ls -l /bin | grep "^-..[sS]"
Here is what to do next
If you followed me through this article, you certainly have realized that knowing some internals about how things are working at the Linux command line, can save you a lot of time and frustration.
And sometimes it’s just fun to leverage these powerful mechanics.
If you wanna know more about such “internal mechanisms” of the Linux command line - written especially for Linux beginners
have a look at “The Linux Beginners Framework”
In this framework I guide you through 5 simple steps to feel comfortable at the Linux command line.
This framework comes as a free pdf and you can get it here.
Wanna take an unfair advantage?
If it comes to working at the Linux command line - at the end of the day it is always about knowing the right tool for the right task.
And it is about knowing the tools that are most certainly available on the Linux system you are currently on.
To give you all the tools for your day-to-day work at the Linux command line, I have created “The ShellToolbox”.
This book gives you everything
- from the very basic commands, through
- everything you need for working with files and filesystems,
- managing processes,
- managing users and permissions, through
- hardware analyses and
- simple shell-scripting to the tools you need for
- doing simple “networking stuff”.
Everything in one single, easy to read book.
With explanations and example calls for illustration.
If you are interested, go to shelltoolbox.com and have a look (as long as it is available).
