Bash

Differential Backup Using Rsync

Rsync is a well known tool for many who work in Linux regularly.  Quite some time ago I put together a short script that uses Rsync and logs the result nicely. The first task Rsync performs is to send an incremental file list which contains any differences between the source and destination drive, then makes changes to the destination drive accordingly.    Because of that last part extreme caution should be used when executing such a script because you run the risk of overwriting information on the destination drive you may not want removed.  I have a dedicated drive for DV backups mounted in /media/dv-backup.  To use this script just change the variables accordingly:

#!/bin/bash

#dv backup script

src=/media/dv 

dest=/media/dv-backup

log=/var/log/backup/dv-backup.log

echo "---" $date "-------------------" >> $log

rsync -t -r -v --delete $src $dest >> $log

echo "--------------- END ---------------" >> $log

I have my backups log in /var/log/backup/ and they merely append the file each time Rsync runs.  I’m aware this is quite a simple script and could even be consolidated to one line.  However, I found that breaking this up makes it easy to read and change.  Also, who wants to remember every Rsync switch they want each time they want to perform a differential backup?

One quick change you could make to make it require less manually editing is replace the variables to take the terminal arguments.  Such a revision is listed below:

src=$1

dest=$2

log=$3

echo "---" $date "-------------------" >> $log

rsync -t -r -v --delete $src $dest >> $log

echo "--------------- END ---------------" >> $log

Usage to achieve same result as original:

./backup.sh /media/dv /media/dv-backup /var/log/backup/dv-backup.log

Both script revisions are attached:  backup.sh backup2.sh Use at your own risk.. don’t blame me if you incorrectly wipe a drive!

Twitter Weather Forecasts Using Linux

twitter sh2

Twitter’s feature to allow updates from terminal allows for a lot of creative ways to use twitter. I used some already written scripts and commands and put it all together to receive weather forecasts for my area at 9am every day on my phone.  Example of SMS message sent to my phone “Forecast for: DES MOINES, IA    Today    Mainly clear and frigid – Low 10“  Here is how I did it:

First I used a written shell script and made output modifications so length would be under Twitter’s length limitations. The modified script is listed below:

Note: I take no credit for the original script. It was available online, and I modified it based on my needs. Dave Taylor is the original author, and offers a book full of shell scripts I recommend checking out! Page Here

#!/bin/sh
# weather - report weather forecast, including lat/long, for zip
# Dave Taylor - found at http://www.intuitive.com/wicked/showscript.cgi?063-weather.sh
# Edited for twitter use by MadHatter (maddhat.com)
llurl="http://www.census.gov/cgi-bin/gazetteer?city=&state=&zip="
wxurl="http://wwwa.accuweather.com"
wxurl="$wxurl/adcbin/public/local_index_print.asp?zipcode="

if [ "$1" = "-a" ] ; then
   size=999; shift
else
   size=5
fi

if [ $# -eq 0 ] ; then
   echo "Usage: $0 [-a] zipcode" >&2
   exit 1
fi

if [ $size -eq 5 ] ; then
   echo ""
   # get some information on the zipcode from the Census Bureau
   lynx -source "${llurl}$1" | \

   sed -n '/^<li><strong>/,/^Location:/p' | \

   sed 's/<[^>]*>//g;s/^ //g'

fi

# the weather forecast itself at accuweather.com

lynx -source "${wxurl}$1" | \
sed -n '/Start - Forecast Cell/,/End - Forecast Cell/p' | \
sed 's/<[^>]*>//g;s/^ [ ]*//g' | \
sed 's/10-Day AccuWeather.com//' | \
sed 's/Today/Today:/' | \
uniq | \
head -$size

exit 0

Save this as weather.sh in your home folder. Then open a text editor and create “tw.sh” saved to your home directory with the following contents:

Note: You need to replace the USER:PASSWORD with the username and password for the secondary twitter account.

#!/bin/sh
#tw.sh
curl -u USER:PASSWORD -d status="$*" http://twitter.com/statuses/update.xml

For this script to run correctly curl must be installed. For ubuntu the command is “sudo apt-get install curl

Now we have the necessary scripts created. To receive updates you will need to create a second twitter account of which to send updates. Make sure you follow this new account with your primary twitter account and turn on device updates if you are looking for update to your mobile device.

All that is left to get things up and running is to add the crontab entry. I added to my sudo crontab by the following “sudo crontab -e” and adding the following entry:

Note: This will run at 9am every day. You can edit yours however you’d like. For testing it may be a good idea to start with “* * * * *” to run every minute to ensure updates are working as planned. Replace “user” with your user name, as well as ZIPCODE with the zip code of the area you would like weather forecasts for.

0 9 * * * /home/user/tw.sh $(/home/user/weather.sh ZIPCODE)

Save the crontab entry. Check that it was added by typing “sudo crontab -l”. That’s all there is to it.

Some useful links for help and customization:

Twitter API, Cron.

Testing Network Performance Using Iperf

Just a quick post on a great tool for testing network performance. Iperf is a very slim tool that will measure TCP/UDP bandwidth performance. It is an available ubuntu package “iperf ” and also has Windows/OSX binaries available. I found a working mirror here. What makes Iperf a good test versus a simple SMB/CIFS transfer for bandwidth performance is that it works in memory versus being limited by hard drive and CPU limitations.

Now a quick guide to get things up and running. The guide will be for two Ubuntu machines, testing network performance between the two.

1.Install – Type the following in terminal “sudo apt-get install iperf ” for both machines

2.Server Setup – Decide upon a server machine and a client. On the server type the following in terminal “iperf -s ”. This will place iperf in server mode, and it will listen for incoming connections.

iperf server

3.Client Setup - On the client machine you have a lot of options to play around with regarding how you want to connect to the server. You can explore what each of these are by typing “iperf –-help ”. The following is a simple test between the server (192.168.2.104), and the client (192.168.2.103).

4.Testing - Type “iperf 192.168.2.104 -i 2” (replacing the IP here with the test server IP and then enter key to start testing. This will connect to the server running on 192.168.2.104 and have an interval of 2 seconds. There will then be 6 tests completed. This should give you all of the information you need regarding your connection speed between the two machines.

iperf client

This should be enough to get you started using iperf. Check out the man page here if you have any usage questions.