Short Tip: replace characters in txt files with sed [Update]

shell.png
When working with txt files or with the shell in general it is sometimes necessary to replace certain chars in existing files. In that cases sed can come in handy:

sed -i 's/foo/bar/g' FILENAME

The -i option makes sure that the changes are saved in the file – in case you are not sure that sed will work as you expect it you should use it without the option but provide an output filename. The s is for “substitute”, the foo is the pattern you are searching the file for, bar is the replacement string and the g stands for “globally” and makes sure that all hits on each line are replaced, not just the first one.

If you have to replace special characters like a dot or a comma, they have to be entered with a backslash to make clear that you mean the chars, not some control command:

sed -i 's/\./\,/g' *txt

Sed should be available on every standard installation of any distribution. At least on Fedora it is even required by core system parts like udev.

Update
I’ve updated the article to be much more precise, thanks to the very detailed and well explained comment from Jeňa Kočí (@cicindel)!

Advertisements

11 thoughts on “Short Tip: replace characters in txt files with sed [Update]”

  1. I’ve grown accustomed to use perl for this kind of thing:

    perl -pi -e ‘s/foo/bar/g’ FILENAMES

    This modifies files in place and allows for reusing matches, e.g.

    perl -pi -e ‘s/G(.*)/K\1/g’

    (maybe ( should be escaped like \(, i’m not quite sure)

  2. I always use something like

    sed -i~ ‘s/foo/bar/’g FILENAME

    which makes a backup of the file with suffix ~. But you can supply nearly everything as a suffix in -i[SUFFIX].

  3. Very confusingly writen:
    -i option doesn’t save into new file, but overwrites the current one *in place*
    s means substitute, not search
    g stands for globally, if you want to remember it easier

    The second example shows the wrong way of doing it, which is stated in the paragraph above. However a lot of people just scan sites for commands and copy them without reading the context, so you should rather show the correct version instead of the wrong one. And especially when even the explanatory text isn’t totally clear: “have to be entered with a backslash” doesn’t make it clear where the backslash needs to be inserted. So the correct way would be:

    sed -i ‘s/\./\,/g’ *txt

    I hope you don’t take my criticism personally. Everyone makes mistakes, including me. I just wanted to point these weaknesses out so other readers don’t get confused.

    Keep up the good work 🙂

    1. Hej Jeňa, thanks for the detailed feedback! And I don’t take it personally, the opposite is the case: I am very thankful for the detailed comment and the ideas how to improve it. I’ve already incorporated your remarks and suggestions and hope that it is now easier to understand and read =)

  4. It should also be made clear that if you are going to remove multiple characters as in . to always backlash each .

    example: sed -i ‘s/\.\.\.\./\,/g’ *txt

    because i tried the example above like this ~ sed -i ‘s/\…./\,/g’ *txt ~ and it ruined the filename.

    hope it helps others 🙂

    Thank You!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s