Useful Snippets & Tools

Last change 17/05/2015

There is a lot of software written for Linux, more than you think (escpecially when switching from the windows world). But it's not always too clear how to use vertain programs or which ones are useful for a certain exercise, and there are some pieces of software that you always have but never quite knew what to do with, so I created a this page to give some useful hints :)

The magic of GREP and SED

Most (*n[iu]x) people know grep.. some even know sed but in combination you can do some nifty stuff.

I recently had following problem:
I got a bunch of html/css/... files where I refer to a bunch of images. Both parts grew over time and now I got a pretts big mess with unreferenced images or references without images. So I wanted a list of all images used in these files.

Using fgrep I found all lines containing png ( fgrep -r png * ) - but that is an ugly mess.. i need to split the lines a little better... that's where sed comes into place. with sed I can stream-search-replace. And in combination with the remove duplicate lines i am set (btw. I  use sed to replace most special chars with a newline, since I don't know how my image paths are used)
perl -e "s/old_string/new_string/g;" -pi.bak $(find . -type f) When working with a lot of images there is also the performance factor.
After reading the xargs, the winner is definitely xargs : it allows mogrify to take advantage of an SMP machine! I have a dual processor machine, so I tell xargs to spawn two mogrify processes (-P 2). In fact, it's "-P 2 -n 1" because for some reason it won't work if you just say "-P 2".

find dir -name '*.tif' -print | xargs mogrify -format png
find . -type d -exec mogrify -format png */*.tif ;

Remove duplicate Lines

A pretty small and REALLY useful script: Remove duplicate lines in a file (or stdin):
#!/usr/bin/perl
# usage: remove_dup_in_place.pl file1 file2 ... fileN
while (<>)
{
  if ($ARGV ne $oldargv)
  {
    open(ARGVOUT, ">$ARGV");
    $oldargv = $ARGV;
    undef %hash;
  }
  $hash{$_}++;
  print ARGVOUT sort keys %hash
     if (eof and defined fileno(ARGVOUT));
}

The Gentoo duplicate package checker

I really like gentoo, but I quickly found out that it just fills my harddrive like nothing (especially with my laptop harddrive which isnt all too big).

I realized that gentoo often installs more than one version of a package - mostly because the oldr version is still required by some other package. But after the "other package" is being updated, it doesnt require the old version any more, but it isnt cleaned out. so I wrote this nifty gentoo script to take care of the finding:
finddupes.tgz
DO NOT JUST UNMERGE ALL THE PACKAGES AS DISPLAYED!!! This MAY break your system!!!! THINK FIST...
tar -xzf finddupes.tgz
cd finddupes
chmod 700 finddupes.sh
./finddupes.sh -h

Image Timestamps

Sometimes when you copy or download images, the timestamp is the current time instead of the time the picture/video has been recorded. This makes sorting them correctly a nuisance. Luckily there are a few tricks that might help:

# photo/video has a format like: IMG_20150421_082631.jpg
ls IMG* | sed 's/IMG_([0-9]*)_([0-9]{4})([0-9]*)/touch -t 12.3 /'
# on some systems the touch command is a little different:
ls IMG* | sed 's/IMG_([0-9]*)_([0-9]{4})([0-9]*)/touch -t 1.23 /'



Site created with Corinis CCM