Author Archives: jan

Backup configuration on every Debian installation

The daily cron job does little bit more, actually.

  • create gzipped tar with /etc and other named directories
  • it creates list of extra installed packages from repositories
  • copy backup over LAN or Internet, checks if NFS disk is mounted or remote directory exists
  • it uses tar, cp and rsync commands

You can add it to cron via this commands:

touch /etc/cron.daily/backup-configuration
chown root:root /etc/cron.daily/backup-configuration
chmod 755 /etc/cron.daily/backup-configuration

/etc/cron.daily/backup-configuration

#!/bin/bash

## global variables

# change variables according to your needs

# destination remote directory to save backups
remote_directory=/disk/backup/home-desktop
# destination local NFS mount point
local_mount_point=/disk
# source local directory to store backups
local_directory=/var/backups
# destination SSH login and server name to save backups
remote_server_login=jan@faix.cz
# list of configuration directories and files you want to backup
backup_list="/etc \
             /usr/local/bin \
             /usr/local/etc \
             /var/spool/cron \
             /var/backups/*.0 \
             /var/backups/*.bak
             /var/backups/*.txt"
# parameter the script was launched with or set default LAN
if [ -z "$1" ]; then
    parameter=-LAN
  else
    parameter="$1"
fi

# do not change following variables
# local server name
machine_name=`hostname -s`

## functions
bck_cfg() {
  aptitude -F %p search "?and(?installed,?not(?automatic))" > $local_directory/pkglist.txt
  tar -czPf $local_directory/$machine_name.tar.gz $backup_list
}
start_job() { printf "\nStarting configuration backup job at `date +%c` ...\n"
}
job_complete() { printf "\nConfiguration backup job successfuly completed on host `hostname` at `date +%c`.\n"
}
dir_not() { printf "\nConfiguration backup job failed on host `hostname` at `date +%c`, $dir_loc does not exist.\n"
}

# check if local_directory does not exist, print error message and exit
if [ ! -d $local_directory ]; then
  dir_loc="local directory $local_directory"
  dir_not && exit
fi

# in case the parameter -LAN or -Internet is used, start configuration backup
case $parameter in
    -LAN)
    ## configuration backup job inside LAN
    dir_loc="local directory $local_directory"
    if [ ! -d $remote_directory ]; then
        mount $local_mount_point
    fi

    if [ -d $remote_directory ]; then
        start_job && bck_cfg
        cp $local_directory/$machine_name.tar.gz $remote_directory
        job_complete
      else
        dir_not
    fi
    ;;
    -Internet)
    ## configuration backup job over Internet
    dir_loc="remote directory $remote_directory"
    if (ssh $remote_server_login "[ ! -d $remote_directory ]"); then
        dir_not_ && exit
    fi

    if [ -d $local_directory ]; then
        start_job && bck_cfg
        rsync -azs --stats $local_directory/$machine_name.tar.gz $remote_server_login:$remote_directory/
        job_complete
      else
        dir_not
    fi
    ;;
    *)
    ## print help
    printf "\nThis script create machine configuration backup and copy it over network to destination server.\n"
    printf "\nConfiguration backup script must be run with one of the floowing parameter:\n"
    printf "\n\t-LAN\t\tbackup configuration within LAN to mounted NFS disk"
    printf "\n\t-Internet\tbackup configuration over Internet via rsync to destination folder\n"
    printf "\t\t\tDon't forget to exchange SSH keys running \"ssh-copy-id $remote_server_login\" from source machine you launch backup configuration script.\n"
    printf "\nCheck variables defined in begining of this script:\n"
    printf "\n\tremote_directory=$remote_directory"
    printf "\n\tlocal_mount_point=$local_mount_point"
    printf "\n\tlocal_directory=$local_directory"
    printf "\n\tremote_server_login=$remote_server_login"
    printf "\n\tbackup_list=$backup_list"
    printf "\n\tmachine_name=$machine_name"
    printf "\n\tdefault parameter=$parameter"
    printf "\n\n"
    ;;
esac

Backup data to remote SFTP server

http://grysz.com/2011/04/04/configure-amazon-s3-backup-with-backupninja-and-duplicity/

Data backup plan:

* pick desired directories with your data you want to backup
* you may backup system directories of your choice (in principle /etc /root /home /usr/local /srv etc.)
* optionally you may backup MySQL database

Used tools (tested on Debian Jessie and Ubuntu 12.04):

* BackupNinja 0.9.10 – for MySQL database backup and backup orchestration
* Duplicity 0.6.23 – filesystem backup
* duplicity-backup – wrapper shell script performing a backup using duplicity

Step 1. Install the tools

apt-get install backupninja duplicity python-paramiko python-lockfile dialog

! Do NOT install Duplicity 0.6.18 from Ubuntu 12.04 default repository, this version has buggy ssh client paramiko library !

Step 2. Configure Backupninja schedule

Check Backupninja configuration (/etc/backupninja.conf) if it suits your needs. The most important parameter is when it controls the backup schedule. I left it with the default value (everyday at 01:00). If you want to get backup report, fill at least `reportemail` parameter value.

Backupninja run every hour from cron job /etc/cron.d/backupninja to launch scheduled backup tasks.

Step 3. Backup MySQL database (optional)

Create a Backupninja backup action for MySQL database dump. The easiest way is to start “ninjahelper” tool and go through create a new backup action/mysql database backup wizard:

Directory to store backups: /var/backups/mysql
Backup all of databases: Yes
MySQL authentication method: password
Create a backup using mysqldump: sqldump
Compress the SQL output files: compress

Ninjahelper creates the action configuration in the /etc/backup.d/20.mysql file.

Perform a test:

backupninja --now --debug

Check if there has been no error messages and if the dump has been created in /var/backups/mysql/sqldump.

Step 4. Backup list of installed packages

echo "aptitude -F %p search \"?and(?installed,?not(?automatic))\" > /var/backups/pkglist.txt" > /etc/backup.d/25.pkglist.sh

The numerical prefix must be lower than last backup action.

Set the owner, group and permissions of new action script:

chown root:root /etc/backup.d/25.pkglist.sh
chmod 700 /etc/backup.d/25.pkglist.sh

Step 5. Exchange SSH key with backup server

From the server, which should be backed up run following commands as `root` user.

* Check if SSH keys already exists: ls ~/.ssh/id_rsa*
* If it doesn’t exist yet, generate them running command: ssh-keygen
* Install root user public key to backup server: ssh-copy-id ue]tod[kinterivznull]ta[xiaf
* Add known host key to known_hosts running: ssh-keyscan -t rsa zviretnik.eu>>~/.ssh/known_hosts

Step 6. Generate a GPG key (optional)

Duplicity encrypts your backups with GPG.

You may also need random number generator for GPG:

apt-get install rng-tools
nano /etc/default/rng-tools
HRNGDEVICE=/dev/urandom
service rng-tools start

If you don’t have the GPG key or you want to create a new one for the backup purposes:

gpg --gen-key
and go though the wizard. Check if it has been created and write down the GPG key ID:

gpg --list-keys

pub 2048R/ 2014-05-27

pub 2048R/44B2DB8E 2014-06-12
Key fingerprint = 9D7E 4D0F 7377 26A7 B5F7 0E27 99E5 B79D 44B2 DB8E
uid Jan Faix
sub 2048R/9413BE41 2014-06-12

Don’t forget to backup your GPG key (~/.gnupg/*.gpg) to safe location!

Step 7. Backup file system and transfer the archive to SFTP server (+optional MySQL backup included)

Install duplicity-backup script.

wget -O /usr/local/bin/duplicity-backup.sh https://raw.githubusercontent.com/zertrin/duplicity-backup/master/duplicity-backup.sh
wget -O /usr/local/etc/duplicity-backup.conf https://raw.githubusercontent.com/zertrin/duplicity-backup/master/duplicity-backup.conf.example

chmod 755 /usr/local/bin/duplicity-backup.sh
chmod 600 /usr/local/etc/duplicity-backup.conf

Check if /usr/local/bin is in PATH environment “echo $PATH”. If not, edit ~/.bash_profile and modify “PATH=$PATH:/usr/local/bin” entry.

Edit duplicity-backup script (/usr/local/bin/duplicity-backup.sh), change the path to configuration file:

CONFIG="/usr/local/etc/duplicity-backup.conf"

Now, edit duplicity-backup script configuration file (/usr/local/etc/duplicity-backup.conf) and fill configuration parameters. The inline documentation is self-explanatory; here are values to adapt:

#AWS_ACCESS_KEY_ID="<your AWS Access Key ID>"
#AWS_SECRET_ACCESS_KEY="<your AWS Secret Access Key>"
ENCRYPTION='yes'
PASSPHRASE="foobar_gpg_passphrase"
GPG_ENC_KEY="foobar_gpg_key"
GPG_SIGN_KEY="foobar_gpg_key"
ROOT="/disk"
# ! Change server hostname !
DEST=ssh://faix@zviretnik.eu//mnt/data/.backups/home-server/
INCLIST=( "/disk/games" )
EXCLIST=( )
STATIC_OPTIONS="--full-if-older-than 7D"
STATIC_OPTIONS="--full-if-older-than 7D --s3-use-new-style --s3-european-buckets"
CLEAN_UP_TYPE="remove-older-than"
CLEAN_UP_VARIABLE="6M"
LOGDIR="/var/log/duplicity/"
LOG_FILE="duplicity-`date +%Y-%m-%d_%H-%M`.log"
LOG_FILE_OWNER="root:root"
VERBOSITY="-v4"
EMAIL_TO=root@faix.cz
EMAIL_SUBJECT="duplicity-backup, `hostname -s`, `date +%Y-%m-%d`"

Test the backup script:

duplicity-backup.sh --backup

ssh faix@zviretnik.eu "ls -la /mnt/data/.backups/home-server/"

You should get the a list similar to this one:

-rw-r--r-- 1 faix users    26255076 Jun 12 22:46 duplicity-full.20140612T204318Z.vol1.difftar.gpg
-rw-r--r-- 1 faix users     4849664 Jun 12 22:46 duplicity-full.20140612T204318Z.vol2.difftar.gpg

Check if you can restore from the backup (quite important issue :)):

mkdir /tmp/restore
duplicity-backup.sh --restore /tmp/restore

If everything works fine, orchestrate duplicity S3 backup from Backupninja. Create a new action (/etc/backup.d/30.backup_to_s3.sh) with following content (/usr/local/bin/duplicity-backup.sh --backup).

The numerical prefix must be higher than of the optional MySQL backup action. In this configuration, first a MySQL dump is created and then, together with other directories (e.g. /etc, /var…), is pushed to the S3 bucket.

Don’t forget to set the owner, group and permissions of new action script:

chown root:root /etc/backup.d/30.backup_to_s3.sh
chmod 700 /etc/backup.d/30.backup_to_s3.sh

Step 8. Test everything together

backupninja --now --debug

and check if there has been no error messages, the backup is in the S3 bucket and it can be restored.

XScreenSaver without keyboard layout indicator workaround

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644534

From: Ilya Sheershoff
Subject: A workaround for keyboard layouts

The following worked for me (Debian Jessie with XFCE 4.10 or Ubuntu 10.04 with GNOME 2.30):

1. Created folder: mkdir ~/.startup
2. And a file: touch ~/.startup/xscreensaver-reset-kbd-us.pl
3. Chmodded it: chmod 755 ~/.startup/xscreensaver-reset-kbd-us.pl
4. Edit (copy&paste the code below): nano ~/.startup/xscreensaver-reset-kbd-us.pl

#!/usr/bin/perl

my $blanked = 0;
open (IN, "xscreensaver-command -watch |");
while (<IN>) {
    if (m/^(BLANK|LOCK)/) {
        if (!$blanked) {
           system "setxkbmap -layout us";
           $blanked = 1;
        }
    } elsif (m/^UNBLANK/) {
        system "setxkbmap -layout us,cz";
        $blanked = 0;
    }
}

5. Add the script to startup applications.

It watches the events from xscreensaver and resets keyboard layout to english only on lock, and adds the cs layout back upon unblank.

For an unknown reason I can’t change the keyboard layout when xscreensaver prompts for password. This helps.

Multisystem on Debian Jessie

If you receive strange error messages upon Multisystem installation on Debian Jessie:

./gui-detect.sh: line 432: /media/Multisystem/.multisystem-test: Permission denied
cat: /media/Multisystem/.multisystem-test: No such file or directory
cat: /tmp/multisystem/multisystem-laisserpasser-usb: No such file or directory
cat: /tmp/multisystem/multisystem-laisserpasser-usb: No such file or directory

than just edit your /etc/fstab and add following line:

/dev/disk/by-label/Multisystem /media/Multisystem         auto    user,umask=0000,uid=1000,gid=100          0       0

Postfix and AntiSpam Smtp Proxy – ASSP

http://www.howtoforge.com/antispam_smtp_proxy

perl -MCPAN -e shell

install Compress::Zlib
install Digest::MD5
install Email::Valid
install File::ReadBackwards
install Mail::SPF::Query
install Mail::SRS
install Net::DNS
install Sys::Syslog
install Time::HiRes

cd /usr/src/
wget http://downloads.sourceforge.net/project/assp/ASSP%20V2%20multithreading/2.4.1%2014085/ASSP_2.4.1_14085_install.zip
unzip ASSP_2.4.1_14085_install.zip

mkdir -p /usr/share/assp/spam
mkdir /usr/share/assp/notspam
mkdir /usr/share/assp/errors
mkdir /usr/share/assp/errors/spam
mkdir /usr/share/assp/errors/notspam

mv -f assp/* /usr/share/assp
rm -fr ASSP_2.4.1* assp changelog.txt Install.txt MacOSX-launchd.txt quickstart.txt Win32-quickstart-guide.txt
chown -R 0.0 /usr/share/assp
cd /usr/share/assp
perl assp.pl

http://hostname:55555

login: root
password: nospam4me

Server Setup
-> Run ASSP as a Daemon (AsADaemon)
-> SMTP Destination -> 127.0.0.1:25
-> Listen Port -> 192.168.122.1:25
Recipients -> Local Domains -> faix.cz
Relaying -> Accept All Mail (acceptAllMail) -> 192.168.122.|127.0.0.1
Security -> Web Admin Password

/etc/postfix/master.cf
localhost:smtp inet n – n – – smtpd

service postfix restart

/etc/init.d/assp

#########################
#!/bin/sh -e

# Start or stop ASSP (Anti-Spam SMTP Proxy)
#
# Script by Abey Marquez &lt;abeymarquez@gmail.com&gt;
# v1.0.1 Changed 'force-reload' to force a restart if it can't reload the config. Also changed 'restart' to start the proc if not running.
# v1.0.0 I'm not an expert but I tried to make this as LSB compliant as possible. Should work really nice with Ubuntu.

### BEGIN INIT INFO
# Provides: ASSP (Anti-Spam SMTP Proxy)
# Required-Start: $syslog, $local_fs
# Required-Stop: $syslog, $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start or stop ASSP
# Description: Start or stop ASSP (Anti-Spam SMTP Proxy)
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
NAME=assp.pl
HOME=/usr/share/assp
DAEMON=$HOME/$NAME
PIDFILE=$HOME/pid
INITSCRIPT=/etc/init.d/assp

. /lib/lsb/init-functions

case "$1" in

start)
log_daemon_msg "Starting ASSP (Anti-Spam SMTP Proxy)" "assp"
start-stop-daemon --start --quiet --pidfile $PIDFILE --startas $DAEMON 2&gt;&amp;1 &gt; /dev/null --chdir $HOME
log_end_msg $?
;;

stop)
log_daemon_msg "Stopping ASSP (Anti-Spam SMTP Proxy)" "assp"
start-stop-daemon --stop --quiet --pidfile $PIDFILE --chdir $HOME
log_end_msg $?
;;

restart)
if [ -f $PIDFILE ]; then
$0 stop
sleep 1
$0 start
else
$0 start
fi
;;

reload)
log_action_begin_msg "Reloading ASSP (Anti-Spam SMTP Proxy) configuration"
if [ -f $PIDFILE ]; then
if kill -1 $(cat $PIDFILE); then
log_action_end_msg 0
else
log_action_end_msg 1
fi
else
log_action_end_msg 1
exit 1
fi
;;

force-reload)
log_action_begin_msg "Reloading ASSP (Anti-Spam SMTP Proxy) configuration"
if [ -f $PIDFILE ]; then
if kill -1 $(cat $PIDFILE); then
log_action_end_msg 0
else
log_action_cont_msg "Could not reload configuration. Restarting"
$0 restart
fi
else
log_action_cont_msg "Could not reload configuration. Restarting"
$0 restart
fi
;;

status)
status_of_proc $DAEMON "ASSP (Anti-Spam SMTP Proxy)"
;;

*)
log_action_msg "Usage: $INITSCRIPT {start|stop|restart|reload|force-reload|status}"
exit 1
;;

esac
exit 0
####################

chmod 755 /etc/init.d/assp
update-rc.d assp defaults

Have fun with all the options and after a week rebuild the bayes database. Check the directories /usr/share/assp/spam and nospam for wrong entries,
if good mail ends up in the spam directory please move it to the nospam directory and vice versa.

Wake-on-LAN (WOL)

https://wiki.archlinux.org/index.php/Wake-on-LAN
http://wiki.xbmc.org/index.php?title=HOW-TO:Set_up_Wake-On-Lan_(Ubuntu)
https://fitzcarraldoblog.wordpress.com/2013/02/26/how-to-prevent-a-usb-mouse-auto-suspending-in-linux-when-a-laptops-power-supply-is-disconnected/

First query the driver to see if it’s defaulted to ‘on’ by using ethtool:
ethtool eth0 | grep Wake-on

To enable the wol feature in the driver, simply run the following:
ethtool -s eth0 wol g

This command does not last beyond the next reboot.

Keep WOL persistent over restarts adding command to following startup script:
/etc/rc.local

ethtool -s eth0 wol g

Test WOL from another computer:
etherwake -i eth0 6c:f0:49:7a:52:10

Sleep computer from CLI:
pm-suspend

In some cases ethtool eth0 shows “Wake-on: g“, but after second suspending of the machine, if magic packet is received, suspend is disabled “Wake-on: d” and doesn’t work anymore.

In my case, I have accidentaly installed laptop-mode-tools on my desktop, so running apt-get purge laptop-mode-tools solved my WOL disabling problem.

Collectd installation on Debian

apt-get install collectd lighttpd php5-cgi rrdtool perl librrds-perl libconfig-general-perl libcurl3-gnutls libsensors4 librrds-perl librrd4 libhtml-parser-perl default-jre-headless libregexp-common-perl

cp -fa /usr/share/doc/collectd/examples/collection3 /var/www/
chown -R www-data:www-data /var/www/collection3/
mkdir /usr/lib/cgi-bin/collectd
cp -fa /usr/share/doc/collectd/examples/collection.cgi /usr/lib/cgi-bin/collectd/

configure cpan
cpan
cpan Config::General
cd /etc/lighttpd/conf-enabled
ln -s 10-cgi.conf ../conf-available/10-cgi.conf
ln -s 10-fastcgi.conf ../conf-available/10-fastcgi.conf
ln -s 15-fastcgi-php.conf ../conf-available/15-fastcgi-php.conf
ln -s 90-javascript-alias.conf /etc/javascript-common/lighttpd.conf

/etc/lighttpd/lighttpd.conf

server.modules = (
"mod_access",
"mod_alias",
"mod_compress",
"mod_redirect",
"mod_rewrite",
)

server.port = 80
server.document-root = "/var/www/"
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )
server.errorlog = "/var/log/lighttpd/error.log"
server.pid-file = "/var/run/lighttpd.pid"
server.username = "www-data"
server.groupname = "www-data"

index-file.names = ( "index.php", "index.html",
"index.htm", "default.htm",
"index.cgi", "index.lighttpd.html" )

url.access-deny = ( "~", ".inc" )

static-file.exclude-extensions = ( ".php", ".pl", ".fcgi", ".cgi" )

dir-listing.encoding = "utf-8"
server.dir-listing = "enable"

compress.cache-dir = "/var/cache/lighttpd/compress/"
compress.filetype = ( "application/x-javascript", "text/css", "text/html", "text/plain" )

include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

/var/www/collection3/index.html

<html>
<head>
<meta HTTP-EQUIV="REFRESH" content="0; url=http://home-server/collection3/bin/index.cgi">
</head>
</html>

/etc/init.d/lighttpd restart

XBMC installation on Debian

XBMC installation:

echo "deb http://www.deb-multimedia.org/ jessie main non-free" >> /etc/apt/sources.list
echo "deb-src http://www.deb-multimedia.org/ jessie main" >> /etc/apt/sources.list
echo "deb http://debian.oppserver.net/xbmc/ jessie main non-free contrib" > /etc/apt/sources.list.d/oppxbmc.sources.list
echo "deb-src http://debian.oppserver.net/xbmc/ jessie main non-free contrib" >> /etc/apt/sources.list.d/oppxbmc.sources.list

wget http://debian.oppserver.net/gpg-debian.oppserver.net-signing-pubkey.asc -O - | apt-key add -

aptitude update

aptitude install xserver-xorg-video-intel xserver-xorg-video-vesa xorg-common x11-session-utils xinit alsa-base nfs-common xbmc xbmc-bin xbmc-data xbmc-eventclients-common xbmc-eventclients-dev xbmc-eventclients-j2me xbmc-eventclients-ps3 xbmc-eventclients-wiiremote xbmc-eventclients-xbmc-send xbmc-skin-confluence xbmc-standalone xbmc-eventclients-common

/etc/systemd/system/xbmc.service

[Unit]
Description = XBMC media center
After = syslog.target

[Service]
User = xbmc
Group = xbmc
Type = simple
ExecStart = /usr/bin/xinit /usr/bin/xbmc-standalone -- :0
Restart = on-failure

[Install]
WantedBy = multi-user.target

systemctl enable xbmc.service

dpkg-reconfigure x11-common
and choose “Anybody”

usermod -a -G adm,sudo,cdrom,floppy,audio,video,plugdev,netdev,fuse xbmc

systemctl start xbmc

Install SuperRepo
http://superrepo.org/get-started/optional-methods/

Install XBMCHub
http://www.xbmchub.com/blog/how-to-use-the-xbmchub-com-configuration-wizard-addon-for-dummies-and-see-mount-rushmore-from-xbmc/