Home Technical Articles AIX Useful SED Oneliners
Useful SED Oneliners Print E-mail
Written by Peter   

Here are some very useful sed one liners, mostly stolen from somewhere.... enjoy!

FILE SPACING:

# double space a file

sed G

# triple space a file

sed 'G;G'

# undo double-spacing (assumes even-numbered lines are always blank)

sed 'n;d'

NUMBERING:

# number each line of a file (simple left alignment). Using a tab (see

# note on '\t' at end of file) instead of space will preserve margins.

sed = filename | sed 'N;s/\n/\t/'

# number each line of a file (number on left, right-aligned)

sed = filename | sed 'N; s/^/ /; s/ *\(.\{6,\}\)\n/\1 /'

# number each line of file, but only print numbers if line is not blank

sed '/./=' filename | sed '/./N; s/\n/ /'

# count lines (emulates "wc -l")

sed -n '$='

TEXT CONVERSION AND SUBSTITUTION:

# IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format

sed 's/.$//'

# IN DOS ENVIRONMENT: convert Unix newlines (LF) to DOS format

sed 's/$//' # method 1

sed -n p # method 2

# delete leading whitespace (spaces, tabs) from front of each line

# aligns all text flush left

sed 's/^[ \t]*//' # see note on '\t' at end of file

# delete trailing whitespace (spaces, tabs) from end of each line

sed 's/[ \t]*$//' # see note on '\t' at end of file

# delete BOTH leading and trailing whitespace from each line

sed 's/^[ \t]*//;s/[ \t]*$//'

# insert 5 blank spaces at beginning of each line (make page offset)

sed 's/^/ /'

# align all text flush right on a 79-column width

sed -e :a -e 's/^.\{1,78\}$/ &/;ta' # set at 78 plus 1 space

# center all text in the middle of 79-column width. In method 1,

# spaces at the beginning of the line are significant, and trailing

# spaces are appended at the end of the line. In method 2, spaces at

# the beginning of the line are discarded in centering the line, and

# no trailing spaces appear at the end of lines.

sed -e :a -e 's/^.\{1,77\}$/ & /;ta' # method 1

sed -e :a -e 's/^.\{1,77\}$/ &/;ta' -e 's/\( *\)\1/\1/' # method 2

# substitute (find & replace) "foo" with "bar" on each line

sed 's/foo/bar/' # replaces only 1st instance in a line

sed 's/foo/bar/4' # replaces only 4th instance in a line

sed 's/foo/bar/g' # replaces ALL instances in a line

# substitute "foo" with "bar" ONLY for lines which contain "baz"

sed '/baz/s/foo/bar/g'

# substitute "foo" with "bar" EXCEPT for lines which contain "baz"

sed '/baz/!s/foo/bar/g'

# reverse order of lines (emulates "tac")

sed '1!G;h;$!d'

# reverse each character on the line (emulates "rev")

sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//'

# join pairs of lines side-by-side (like "paste")

sed 'N;s/\n/ /'

SELECTIVE PRINTING OF CERTAIN LINES:

# print first 10 lines of file (emulates behavior of "head")

sed 10q

# print first line of file (emulates "head -1")

sed q

# print last 10 lines of file (emulates "tail")

sed -e :a -e '$q;N;11,$D;ba'

# print last line of file (emulates "tail -1")

sed '$!d'

# print only lines which match regular expression (emulates "grep")

sed -n '/regexp/p' # method 1

sed '/regexp/!d' # method 2

# print only lines which do NOT match regexp (emulates "grep -v")

sed -n '/regexp/!p' # method 1, corresponds to above

sed '/regexp/d' # method 2, simpler syntax

# print 1 line of context before and after regexp, with line number

# indicating where the regexp occurred (similar to "grep -A1 -B1")

sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h

# grep for AAA and BBB and CCC (in any order)

sed '/AAA/!d; /BBB/!d; /CCC/!d'

# grep for AAA or BBB or CCC (emulates "egrep")

sed -e '/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d

# print only lines of 65 characters or longer

sed -n '/^.\{65\}/p'

# print only lines of less than 65 characters

sed -n '/^.\{65\}/!p' # method 1, corresponds to above

sed '/^.\{65\}/d' # method 2, simpler syntax

# print section of file from regular expression to end of file

sed -n '/regexp/,$p'

# print section of file based on line numbers (lines 8-12, inclusive)

sed -n '8,12p' # method 1

sed '8,12!d' # method 2

# print line number 52

sed -n '52p' # method 1

sed '52!d' # method 2

sed '52q;d' # method 3, efficient on large files

# print section of file between two regular expressions (inclusive)

sed -n '/Iowa/,/Montana/p' # case sensitive

SELECTIVE DELETION OF CERTAIN LINES:

# print all of file EXCEPT section between 2 regular expressions

sed '/Iowa/,/Montana/d'

# delete duplicate lines from a sorted file (emulates "uniq"). First

# line in a set of duplicate lines is kept, the rest are deleted

sed '$!N; /^\(.*\)\n\1$/!P; D'

# delete ALL blank lines from a file (same as "grep '.' ")

sed '/^$/d'

# delete all CONSECUTIVE blank lines from file except the first; also

# deletes all blank lines from top and end of file (emulates "cat -s")

sed '/./,/^$/!d' # method 1, allows 0 blanks at top, 1 at EOF

sed '/^$/N;/\n$/D' # method 2, allows 1 blank at top, 0 at EOF

# delete all CONSECUTIVE blank lines from file except the first 2:

sed '/^$/N;/\n$/N;//D'

# delete all leading blank lines at top of file

sed '/./,$!d'

# delete all trailing blank lines at end of file

sed -e :a -e '/^\n*$/N;/\n$/ba'

SPECIAL APPLICATIONS:

# remove nroff overstrikes (char, backspace) from man pages

sed "s/.`echo \\\b`//g" # double quotes required for Unix environment

sed 's/.\x08//g' # hex expression for GNU sed (octal is "\010")

# get Usenet/e-mail message header

sed '/^$/q' # deletes everything after first blank line

# get Usenet/e-mail message body

sed '1,/^$/d' # deletes everything up to first blank line

# get Subject header, but remove initial "Subject: " portion

sed '/^Subject: */!d; s///;q'

# get return address header

sed '/^Reply-To:/q; /^From:/h; /./d;g;q'

# parse out the address proper. Pulls out the e-mail address by itself

# from the 1-line return address header (see preceding script)

sed 's/ *(.*)//; s/>.*//; s/.*[:<] *//'

# add a leading angle bracket and space to each line (quote a message)

sed 's/^/> /

# delete leading angle bracket & space from each line (unquote a message)

sed 's/^> //'

# remove most HTML tags (accommodates multiple-line tags)

sed -e :a -e 's/<[^<]*>/ /g;/</{N;s/\n/ /;ba;}'

# extract multi-part uuencoded binaries, removing extraneous header

# info, so that only the uuencoded portion remains. Files passed to

# sed must be passed in the proper order. Version 1 can be entered

# from the command line; version 2 can be made into an executable

# Unix shell script. (Modified from a script by Rahul Dhesi.)

sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode # vers. 1

sed '/^end/,/^begin/d' $* | uudecode # vers. 2

# zip up each .TXT file individually, deleting the source file and

# setting the name of each .ZIP file to the basename of the .TXT file

# (under DOS: the "dir /b" switch returns bare filenames in all caps).

echo @echo off >zipup.bat

dir /b *.txt | sed "s/^\(.*\)\.TXT/pkzip -mo \1 \1.TXT/" >>zipup.bat

How to print all lines from file up to a line that ends with 'STOP':

cat file | sed '/STOP$/q'

How to print lines 4 thru 10 from a file:

cat file | sed -n '4,10p'

How to globally replace a string in a file:

cat file.old sed '/oldstring/s//newstring/g' > file.new

How to remove blank lines from a file:

cat file | sed '/^$/d' > file.new

Search for a string using an embedded wildcard..print any line that

contains 'Thom' followed by any number of characters, followed by 'Ros':

cat /etc/passwd | sed -n '/Thom.*Ros/p'

To do the OPPOSITE of the above (note the ! infront of the 'p', and

how it must be escaped with '\'):

cat /etc/passwd | sed -n '/Thom.*Ros/\!p'

Last Updated on Sunday, 24 January 2010 09:40
 
SoluFlex

Login Form



Who's Online

We have 2 guests online

Statistics

Members : 149
Content : 37
Web Links : 6
Content View Hits : 38790