I do a lot of work on my servers (like webpage updates) and normally I ssh into them and then use vi/mcedit/nano to work on my stuff... It also bothered me that I cannot use tools like nvu or similar that would greatly improve my speed. I used to install X on my servers so I could open remote X Programs, but thats a huge security flaw and even the updates were driving me nuts.. until yea.. I found out about shfs ... mounting remote filesystems using only ssh... here is an outtake from the gentoo wiki :

Similar to sftp, but actually mounts the remote files as a filesystem using only ssh! In addition to being secure, this makes it the most versatile and easy to use *nix network filesystem ever. It is usable on both public and private networks, and absolutely no configuration is required once you have sshd running on the remote machine (as well you should!) (Note: Only one command can be executed at a time! So while you are copying a big file for example, you may not be able to copy a second at the same time, or make a directory listing...)

On the Server /etc/init.d/sshd start
rc-update add sshd default On the client emerge shfs
modprobe shfs
# Automatically load shfs
echo "shfs" >> /etc/modules.autoload To mount mkdir /mnt/shfs
shfsmount username@server:/path/to/any/directory /mnt/shfs
-- or equivalently --
mount -t shfs username@server:/path/to/any/directory /mnt/shfs


wget is an extremely useful tool for mirroring or downloading from the web (there are windows builds too), when it comes to f.e. backup a dynamic created page (with ?id=,...) it is also extremely nice to you.

What this does the following: mirror the page (-m), rewrite all links (-k), tread all dynamic files as html (-E makes even sure that asdf.cgi?id=asdf will be converted to asdf.cgi?id=asdf.html) and make sure we can view everything under windows, too (--restrict-file-names=windows).

When having style sheets with images (f.e. as background) you may have to manually edit the path - this is pretty much the only thing wget does not do for you. wget -m -E --restrict-file-names=windows -k http://domain.tld/path

I really like gentoo, but I quickly found out that it just fills my harddrive like nothing (especially with my laptop harddrive which isnt all too big).

I realized that gentoo often installs more than one version of a package - mostly because the oldr version is still required by some other package. But after the "other package" is being updated, it doesnt require the old version any more, but it isnt cleaned out. so I wrote this nifty gentoo script to take care of the finding:
finddupes.tgz
DO NOT JUST UNMERGE ALL THE PACKAGES AS DISPLAYED!!! This MAY break your system!!!! THINK FIST...
tar -xzf finddupes.tgz
cd finddupes
chmod 700 finddupes.sh
./finddupes.sh -h

Sometimes you have a lot of files, and in every file something has to be changed. Under windows I used the replace in files feature from UltraEdit. But under linux, it works much smoother and faster.

The following example walks through all subdirectories and files and changes old_string to new_string in each of them. Even creating a nice .bak file (if you do not want any .bak files to be created just remove the .bak). You can use any regexpression in the "" after the -e. perl -e "s/old_string/new_string/g;" -pi.bak $(find . -type f)

Most (*n[iu]x) people know grep.. some even know sed but in combination you can do some nifty stuff.

I recently had following problem:
I got a bunch of html/css/... files where I refer to a bunch of images. Both parts grew over time and now I got a pretts big mess with unreferenced images or references without images. So I wanted a list of all images used in these files.

Using fgrep I found all lines containing png ( fgrep -r png * ) - but that is an ugly mess.. i need to split the lines a little better... that's where sed comes into place. with sed I can stream-search-replace. And in combination with the remove duplicate lines i am set (btw. I  use sed to replace most special chars with a newline, since I don't know how my image paths are used)
perl -e "s/old_string/new_string/g;" -pi.bak $(find . -type f)When working with a lot of images there is also the performance factor.
After reading the xargs, the winner is definitely xargs : it allows mogrify to take advantage of an SMP machine! I have a dual processor machine, so I tell xargs to spawn two mogrify processes (-P 2). In fact, it's "-P 2 -n 1" because for some reason it won't work if you just say "-P 2".

find dir -name '*.tif' -print | xargs mogrify -format png
find . -type d -exec mogrify -format png */*.tif ;

A pretty small and REALLY useful script: Remove duplicate lines in a file (or stdin):
#!/usr/bin/perl
# usage: remove_dup_in_place.pl file1 file2 ... fileN
while (<>)
{
  if ($ARGV ne $oldargv)
  {
    open(ARGVOUT, ">$ARGV");
    $oldargv = $ARGV;
    undef %hash;
  }
  $hash{$_}++;
  print ARGVOUT sort keys %hash
     if (eof and defined fileno(ARGVOUT));
}

Sometimes you have a lot of files, and in every file something has to be changed. Under windows I used the replace in files feature from UltraEdit. But under linux, it works much smoother and faster.

The following example walks through all subdirectories and files and changes old_string to new_string in each of them. Even creating a nice .bak file (if you do not want any .bak files to be created just remove the .bak). You can use any regexpression in the "" after the -e. perl -e "s/old_string/new_string/g;" -pi.bak $(find . -type f)

If you want to delete files with a specific name (f.e. all *.bak) which are hidden within the subdirectory structure, you use the rm in combination with the find command.

The following example deletes all bak files. If you want to delete something else just exchange the *.bak with whatever fits your needs (f.e. CVS). The -rf makes sure even whole subdirectories are deleted, so be careful!!! find . -name *.bak -exec rm -rf {} ;

Mogrify is a pretty useful program used by many webapplications. It allows the modification of images, like resize, crop, convert.
While supporting a big portal (the teendays event), I found out that most thumbnails created (they were only max 100x100pixels) where a few kilobytes big... too big in my opinion. I found out that the problem is the profile stored in the images, after removing the profile with the option "+profile *" the files shrank to about 1k - 2-4k smaller than before without quality loss!!!

Remember imagemagick and therefore mogrify is also available for windows!
# create a nice thumbnail 100x100 max (relation taken from original)
mogrify +profile "*" " -size 100x100 -resize 100x100 somepic.jpg

Converting youtube videos consists of two parts: first you need to get the video from there. Best to use http://www.arrakis.es/~rggi3/youtube-dl/ for this. its a little python script that just does what it says and this real nice.

# get the youtube video and rename the file nicely
youtube-dl -t "http://www.youtube.com/watch?v=XXXXX"

once you got them you need to convert them to avi and - since most of the time the bigger videos are split into many files you can merge those avis to one.

# if you downloaded more than one flv hereis a convenient line to reencode all of the flvs into avi
ls | sed 's/.flv//g' | xargs -n 1 -i ffmpeg -i '{}'.flv -ab 56 -ar 22050 -b 500 -s 320x240 -vcodec xvid -acodec mp3 '{}'.avi