A Few Unix / Linux Tips and Tricks

Command History Editing in csh and tcsh

Just add a line to your file $HOME/.cshrc saying:

bindkey -v

Go get a copy of tcsh if you don't already have one. It is a vastly improved and expanded version of csh. It is free yet very well supported like most really great Unix stuff. Don't worry, everything you do now in csh will work exactly the same. There will just be a lot more things you can use as you learn how. Hint:
man tcsh

By the way, the name tcsh is obviously formed by adding a t to csh, but why a t? According to the manual page, as a tribute to the TENIX and TOPS-20 operating systems, which had some great features that were added during the csh-to-tcsh improvement.

Automatic Filename Completion

You want to accomplish something with the computer beyond just practicing your typing. Let the shell finish your commands and filenames — it's much faster it an makes no errors! The tcsh command does this by default, but you can add this to your file $HOME/.cshrc to tell csh to also complete file and command names when you press the Tab key:

set filec 

If the choice is ambiguous because there are multiple possibilities based on what you have typed so far, the shell will beep at you. That is, unless you also set autolist, in which case the shell shows you the choices. This is like the bash behavior, and it is available in tcsh but not csh

set filec
set autolist 

If you also set addsuffix then the autolist distinguishes between files and directories by adding "/" to the names of directories.

set filec
set autolist
set addsuffix 

You could also have the tcsh shell offer suggestions when you mis-type commands. I don't like this — I should have typed it correctly in the first place and filename and command completion should have avoided this problem. But if you want to:

set correct=cmd 

Avoid file removal and overwriting disasters

In both csh and tcsh the rmstar variable will cause a command like rm * to ask if you really want to remove everything. One of the few times that DOS default behavior might be especially useful, here's how to turn it on in your $HOME/.cshrc file:

set rmstar 

You will want to do that in interactive shells only, as it does not progress until you press y or n and a script would hang endlessly

Overwriting is when you redirect output into a file that already exists:

% some-command > valuable-data 

or when you move or copy a file on top of pre-existing valuable data:

% cp foo > valuable-data-1
% mv foo > valuable-data-2 

You can prevent the first disaster with the noclobber shell variable, and the second form of disaster with aliases for cp and mv. All this goes in your $HOME/.cshrc file:

# Don't overwrite existing files with stdout or stderr streams
set noclobber
# Don't overwrite existing files with "cp" or "mv"
alias cp   'cp -i'
alias mv   'mv -i' 

You can override these if you really want to overwrite the file:

% some-command >! valuable-data-1
% cp -f foo valuable-data-2
% mv -f foo valuable-data-3 

Show the Current Directory in the Prompt

My $HOME/.cshrc file contains something like this:

set host=`uname -n | sed 's/\..*//'`
set domain=`uname -n | sed 's/^[^\.]*//'`
set os=`uname -s`
set prompt0="${os}:${host}"
set prompt="${prompt0}:`echo $cwd | sed 's@'${HOME}'@~@'` % "
alias cd 'cd \!* ; set prompt="'${prompt0}':`pwd | sed "s@'$HOME'@~@"` % "' 

That makes my prompt contain:

Linux:penguin:~/src % _

OpenBSD:berkeley:/usr/share/doc % _ 

Put the Hostname and Directory in the xterm Title Bar

Exactly how you go about this depends on the interactive shell you're using. You'll probably set up a function (for ksh) or an alias (for csh or tcsh) or use the cwdcmd special alias (for tcsh). Whatever. The main trick is, whenever you do a cd your shell should also utter the following arcane incantation:

echo -n "^[]2;${HOST}:$cwd^G^[]1;${HOST}^G"

Yikes! You need to have the variable HOST set to the hostname, and if your shell insists on using CWD or pwd or PWD to indicate the current working directory, change "$cwd" above accordingly. Finally, ^[ really corresponds to control-left-bracket, which happens to be the same as the Escape key, and ^G really corresponds to a control-G. That incantation puts hostname:path in the xterm titlebar, and associates simply hostname with its icon. Tune what you want to put where accordingly.

Yeah, that's a real mess. That's why (a) I just define it in my shell start-up file, (b) I can never remember exactly how to do it without looking there, and (c) I never try to explain this to anyone, but tell them to look at this page. Now you see why.

Getting Rid of the Heinous "Caps Lock" Key

OK, so this is just a pet peeve of mine, but maybe the caps lock key annoys you, too.

If you are the system administrator and want to fix this system wide, put the following in a boot script like /etc/rc.d/rc.local

A user could put it in their $HOME/.login file:

xmodmap -e "remove Lock = Caps_Lock"
xmodmap -e "add Control = Caps_Lock"

Where Things Go

A common question is, "Explain to me where everything is located in Unix!" or, conversely, "Explain the purpose and syntax of every file under /etc. And while you're at it, /var and /usr/share as well!"

Oof! The only useful polite response is to send them to the Filesystem Hierarchy Standard: http://www.pathname.com/fhs/

Add the Great Free Stuff Sun Left Out of Solaris

http://sunfreeware.com/ seems to be run by Sun. It's a place where you can get the software but you can't legitimately pester Sun for support.

Some awk Tricks

You can do just about anything with awk and it's a standard part of Unix. Yes, Perl does more, but it's not quite as universally available, and due to the difficulty of reliably determining the purpose of Perl code from reading it, some organizations have administratively decided against its use for mission-critical applications. Here's an awk example I use in class. Imagine that a database contains expense records, and each record is printed in a line like in the following extract:

 5.44  meal       Feb 27 1998  Chicago  Breakfast before meeting
14.07  meal       Feb 27 1998  Chicago  Dinner after meeting
 0.75  transport  Feb 27 1998  Chicago  Toll driving home

Here's an awk script to print detailed records and summaries:

BEGIN	{	# This stuff happens before line 1!
		printf("\nExpense Report\n");
		printf("======================================\n");
	}

	{	# No pattern -- this matches and runs for every line!
		printf("Date: %s %s, %s\n", $3, $4, $5);
		printf("  City: %s\n", $6);
		printf("  Item: %s", $2);
		if (NF > 6) {
			printf("(");
			for (i = 7; i <= NF; i++)
				printf(" %s", $i);
			printf(")");
		}
		printf("\n");
		printf("  Cost: %s\n", $1);
		total_cost += $1;
		num_receipts ++;
	}

END	{	# This stuff happens after the last line!
		printf("======================================\n");
		printf("%d reports handled\n", num_receipts);
		printf("Total amount: $%.2f", total_cost);
		printf("  (average cost = $%.2f)\n", num_receipts/total_cost);
	}

Store the above awk program in a file. Let's say we name the file summary.awk (not that there's anything magic about the ".awk" extension, it's just a reminder). Now, if the database is stored in the file expenses, you can process the file as:

$ awk -f summary.awk expenses

How about something useful for processing WWW server logs? Your httpd logs will contain lists of domain names. If you simply sort them, they will be put in lexicographic ASCII order:

betelgeuse.ecn.purdue.edu
cleveland.widget.com
dover.gadget.com
elephant.elvis.com
happy.gadget.com
larry.elvis.com
methuselah.elvis.com
narvik.vikingnet.no
ollie.cs.purdue.edu
pc12.ecn.purdue.edu
pc17.amish.org
pc22.widget.com
sam.gadget.com
thor.library.purdue.edu
ws17.widget.com
www.gadget.com
www.purdue.edu
www.amish.org
www.widget.com
zaphod.cs.purdue.edu
zaphod.ecn.purdue.edu
zaphod.gadget.com
zaphod.library.purdue.edu

That's a sorted list, but we would prefer that the domains sorted right-to-left. We can't easily do that with sort since they have varying numbers of fields. We would like the com names first, then the edu names, then the no names (you did notice the Norweigan hostname, right?), then the org names.

Within each top-level domain, we would like to sort them by second-level domain.

Within each second-level domain, we would like to sort them by the third-level domain, and so on. We want to get this list, complete with indentation making it easy to read:

	       elephant.elvis.com
	          larry.elvis.com
	     methuselah.elvis.com
	         dover.gadget.com
	         happy.gadget.com
	           sam.gadget.com
	           www.gadget.com
	        zaphod.gadget.com
	     cleveland.widget.com
	          pc22.widget.com
	          ws17.widget.com
	           www.widget.com
	      ollie.cs.purdue.edu
	     zaphod.cs.purdue.edu
	betelgeuse.ecn.purdue.edu
	      pc12.ecn.purdue.edu
	    zaphod.ecn.purdue.edu
	  thor.library.purdue.edu
	zaphod.library.purdue.edu
	           www.purdue.edu
	      narvik.vikingnet.no
	           pc17.amish.org
	            www.amish.org

No problem! Just apply this shell script to the file log:

#!/bin/sh

awk -F. '{ for (i = NF; i > 0; i++)	# Reverse fields, remove dots
		printf("%s ", $i);
	   printf("\n"); }' log  |  \
sort |  \				# Sort
awk '{	printf("%s", $NF);		# Reverse fields again, re-insert dots
	for (i = NF-1; i >= 0; i--)
		printf(".%s", $i);
	printf("\n"); }'  |  \
awk '{ printf("%30s\n", $1); }		# Right-justify the output

The first awk prints each domain names in reverse order with the dots removed. The sort puts the reversed names into our preferred order. The second awk reverses the domains again, and re-inserts the dots. The third awk formats the output with printf() to make the complete names line up along the right margin.

Gratuitous Pop-Culture References to Unix

$ sed -e 's/a/akka/g' \
	-e 's/A/Akka/g' \
	-e 's/o/ulla/g' \
	-e 's/O/Ulla/g' english-story > finnish-story

Where next?


Home Page Unix/Linux TCP/IP Infosec Travel Radio Site Map Contact
Use /bin/vi! Manipulate images with ImageMagick! Hosted on OpenBSD
Hosted on Apache Valid XHTML 1.1! Valid CSS!
© Bob Cromwell Nov 2008. Created with /bin/vi and ImageMagick, hosted on OpenBSD with Apache.    Root password available here