Tag Archives: cron

Debian server

Články , , , , , , , , , , , , , , ,

I use my home Debian server as router, firewall, NAS and HTPC.

I have installed system on SSD drive and data are on two LUKS encrypted WDC 2TB drives in RAID1.

System in installed as Debian basic server, the rest of the packages was installed and configured the way as it follows.

System drive: 32 GB ADATA SP800 (SSD)
Data drive: 2x 2 TB WDC WD20EARX (RAID1)

Complete HTPC and NAS hardware configuration

Table of content

Restore installed packages from package list
XBMC
RAID1
LVM
Luks encryption
Mounted file systems
Encrypted file systems
Localisations
User defaults
Umask
Users and groups
Sudo
Linux kernel settings
NTP server
Time zone
Network interfaces
UFW firewall
Dnsmasq (DNS and DHCP server)
Squid
OpenVPN server
NFS server
Postfix
Spamassassin
Postgrey
Sasl
CUPS
Cron
Sarg
Rsyslog
Logrotate
Duplicity and BackupNinja
Rsnapshot
Rsync
Rdiff-backup
Lighttpd and Collectd
Fail2ban
Logwatch
OSSEC
Transmission-daemon
Monit
Ntopng
LinuxCounter
DynDns
Random generator
Sensors
SmartmonTools
HDD temperature
Safe LUKS headers

List of installed services and configuration

Restore installed packages from package list.

xargs aptitude --schedule-only install < pkglist.txt && aptitude install

XBMC

/etc/apt/sources.list.d/xbmc-ffmpeg.list

# XBMC for Wheezy, it also needs packages from wheezy-backports
deb https://people.debian.org/~rbalint/ppa/xbmc-ffmpeg xbmc-ffmpeg-wheezy-backports/
deb http://http.debian.net/debian wheezy-backports main contrib non-free

apt-get update && apt-get install xbmc... follow XBMC installation on Debian

RAID1

apt-get install mdadm

parted /dev/sd{b,c} mklabel msdos (or gpt if you have drive bigger than 2 TB)
parted /dev/sd{b,c} mkpart primary raid 1 -1
mdadm -v --create /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1
cat /proc/mdstat

LVM

Create logical volume (LVM) on top of existing RAID1.

pvcreate /dev/md0
vgcreate vg_disk /dev/md0
lvcreate -L 2t -n lv_data vg_disk

Create logical volumes for {root,home,var/log,temp}.

parted /dev/sda mklabel msdos
pvcreate /dev/sda
vgcreate vg_system /dev/sda
lvcreate -L 7g -n lv_root /dev/vg_system/lv_{root,home,var/log,temp}

Luks encryption

Create encrypted disk on top of logical volume (lv_data) created above.

apt-get install cryptsetup

cryptsetup --verify-passphrase --verbose --hash=sha256 --cipher=aes-xts-plain --key-size=512 luksFormat /dev/vg_disk/lv_data
cryptsetup luksOpen /dev/vg_disk/lv_data disk
mkfs.ext4 -L Data -m 1 -O dir_index,filetype,sparse_super /dev/mapper/disk
mkdir /disk
mount /dev/mapper/disk /disk

Mounted file systems

My /etc/fstab layout on system drive.

Encrypted file systems

My /etc/crypttab file, no automount encrypted drive.

Localisations

/etc/locale.gen

cs_CZ.UTF-8 UTF-8
en_US.UTF-8 UTF-8
th_TH.UTF-8 UTF-8

/etc/default/locale

LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_ALL=en_US.UTF-8

locale-gen

User defaults

/etc/default/useradd

SHELL=/bin/bash
GROUP=100
HOME=/home
SKEL=/etc/skel
CREATE_MAIL_SPOOL=yes

Umask

Add following line.
/etc/pam.d/common-session

session optional pam_umask.so

Change default umask.
/etc/login.defs
UMASK    007

Users and groups

Add existing user to additional groups.
usermod -a -G adm,sudo,users username
Check content of passwd and group file and do necessary adjustments.
vipw [-s]
vigr [-s]
/etc/passwd

...
username:x:1000:1000::/home/username:/bin/bash

/etc/group
...
sudo:x:27:username
users:x:100:username,username1
username:x:1000:

Sudo

visudo
Change existing line to disable password confirmation when using sudo. I don’t like typing passwords, you may leave it in default.

%sudo ALL=(ALL) NOPASSWD:ALL

Linux kernel settings

/etc/sysctl.conf

NTP server

apt-get install ntp
Change your network address, mask and broadcast accordingly.
/etc/ntp.conf

Time zone

/etc/timezone

Europe/Prague

Network interfaces

/etc/network/interfaces

ASUS P8H61-M PRO build-in NIC
WAN network, interface eth0
MAC 10:bf:48:7a:42:cf
RTL8111/8168B (rev 06)

AXAGO PCEE-GR
LAN network, interface eth1
MAC 00:13:3b:10:1a:d4
RTL8111/8168B (rev 07)

Both card are using identical chipset Realtek RTL8111E.

AXAGO PCEE-GR gigabit ethernet PCI Express adapter is excellent, but cheap czech product which cost only $10.

UFW firewall

apt-get install ufw

Filter UFW messages from /var/log/syslog and log them only to /var/log/ufw.log.

/etc/rsyslog.d/20-ufw.conf

:msg,contains,"[UFW " /var/log/ufw.log
& ~

/etc/default/ufw

DEFAULT_FORWARD_POLICY="ACCEPT"

/etc/ufw/sysctl.conf

net/ipv4/ip_forward=1

/etc/ufw/before.rules

If you have existing set of rules, you can copy it from location below or use mine. The best way is to define your own rules using UFW. Or you can setup your firewall and router server following this guide.

/lib/ufw/user.rules

Dnsmasq (DNS and DHCP server)

apt-get install dnsmasq

/etc/dnsmasq.conf

touch /var/log/dnsmasq.log /var/lib/misc/dhcp.leases
chown dnsmasq /var/log/dnsmasq.log /var/lib/misc/dhcp.leases

/etc/resolv.conf

nameserver 127.0.0.1

/etc/resolv.dnsmasq

domain homelan
search homelan
nameserver 8.8.8.8
nameserver 8.8.4.4

chown dnsmasq /etc/resolv.dnsmasq

/etc/hosts
/etc/logrotate/dnsmasq

Squid

Setup Squid 3.4.8 transparent proxy.

For more about Squid transparent proxy installation and configuration you can read here.

OpenVPN server

apt-get install openvpn bridge-utils

/etc/network/interfaces
/etc/openvpn/easy-rsa/vars
/etc/openvpn/server.conf

ufw allow 1194

For more about OpenVPN installation and configuration you can read here.

NFS server

apt-get install nfs-kernel-server nfs-common portmap

Modify following parameters in the files below. You can read more about NFS installation here.

/etc/default/nfs-common

NEED_IDMAPD=YES

/etc/default/nfs-kernel-server

RPCMOUNTDOPTS="-p 13030"

/etc/exports

/disk 192.168.122.0/27(rw,no_root_squash,async,no_subtree_check,crossmnt,fsid=0)

Postfix

/etc/postfix/main.cf

/etc/postfix/master.cf

/etc/postfix/sender_access

/etc/postfix/virtual

/etc/aliases

/usr/local/bin/spamfilter.sh

/etc/postfix/canonical

arpwatch@home-router    arpwatch@faix.cz
root@home-router.homelan  root@faix.cz
ossecm@home-server    ossecm@faix.cz

/etc/postfix/generic

@faix.homelinux.net @faix.cz

/etc/postfix/rbl_override

77.75.76.26 OK
82.192.75.110 OK
ns35.mailserver.com OK

/etc/postfix/recipient_canonical

/^(.*)@(.*)\.homelan$/     ${1}.${2}@faix.cz
/^(.*)@(.*)\.localhost$/     ${1}.${2}@faix.cz

/etc/postfix/sender_canonical

/^(.*)@(.*)\.homelan$/       ${1}@faix.cz
/^(.*)@(.*)\.home-*$/        ${1}@faix.cz
/^(.*)@(.*)\.localhost$/     ${1}@faix.cz

/etc/postfix/sasl/smtpd.conf

pwcheck_method: saslauthd
mech_list: plain login

/etc/mailname

faix.cz

passwd mail
mkdir /var/spamchk

/etc/ssl/private/smtpd.key
/etc/ssl/certs/cacert.pem
/etc/ssl/certs/smtpd.crt

How to generate your certificate, see more in Sasl section.

For more about Postfix installation and configuration you can read here.

Spamassassin

apt-get install spamassassin spamc

/etc/spamassassin/local.cf

/etc/default/spamassassin

ENABLED=1
SAHOME="/var/log/spamassassin/"
OPTIONS="--create-prefs --max-children 2 --username debian-spamd -H ${SAHOME} -s ${SAHOME}spamd.log"
PIDFILE="/var/run/spamd.pid"
CRON=1

For more about Spamassasin installation and configuration you can read here.

Postgrey

/etc/defaults/postgrey

POSTGREY_OPTS="--inet=192.168.122.1:10023 --delay=60"

mkdir /var/log/spamassassin/

For more about Postgrey installation and configuration you can read here.

Sasl

apt-get install sasl2-bin libsasl2-2 libsasl2-modules

/etc/default/saslauthd

Don’t forget to add postfix to group sasl.
adduser postfix sasl

For more about Postfix SASL configuration you can read here.

CUPS

apt-get install cups cups-common cups-pdf

/etc/cups/cupsd.conf
/etc/cups/printers.conf

For more about CUPS configuration you can read here.

Cron

/etc/cron.d/reminder
/etc/cron.d/openbl
/etc/cron.daily/backup-configuration

/etc/crontab
Regularly launched various reminders, backups etc.

# m h dom mon dow user  command
50 1  *  *  *  root    /usr/local/bin/backup-pm-wol.sh

/usr/local/bin/backup-pm-wol.sh

#!/bin/sh
etherwake -i eth1 6c:f0:49:7a:52:10

Sarg

Install Sarg – Squid Analysis Report Generator to see the usage of Internet.

For more about Sarg report generator installation and configuration you can read here.

Rsyslog

Configure remote syslog server to capture log messages from important devices on your LAN.

mkdir /var/log/rsyslog
chown root:adm /var/log/rsyslog

/etc/rsyslog.d/90-rsyslog.conf
/etc/rsyslog.d/20-ufw.conf
/etc/rsyslog.conf

/etc/logrotate.d/rsyslog

#insert right after first line /var/log/syslog
#to rotate logs from remote syslog
/var/log/rsyslog/*/*.log

For more about Remote syslog configuration you can read here.

Logrotate

Don’t let your logs grow over reasonable limits. Here are some customised examples.

/etc/logrotate.d/rsyslog
/etc/logrotate.d/dnsmasq
/etc/logrotate.d/fail2ban
/etc/logrotate.d/monit

Duplicity and BackupNinja

/etc/backup.d/30.backup_filesystem.sh
/etc/backup.d/25.pkglist.sh
/usr/local/bin/duplicity-backup.sh
/usr/local/etc/duplicity-backup.conf
/etc/backupninja.conf

mkdir /var/log/duplicity
ln -s /root/.cache /disk/.cache

For more about Duplicity and BackupNinja configuration you can read here.

Rsnapshot

Local LAN data backups are done every day using rsnapshot which is configured and run from home-desktop to do snapshots to local RAID1 mirror (/mnt/mirror/.snapshots) from remote home-server NFS export (/disk).

/etc/crontab
/usr/local/bin/backup-rsnapshot.sh
/etc/rsnapshot.conf
/etc/rsnapshot.d/rsync-exclude.txt

Rsync

Local LAN ISO backup is done every day using rsync which is configured and run from home-desktop to sync to local RAID1 mirror “/mnt/mirror/iso” from remote home-server NFS export “/disk/iso“.

/etc/crontab
30 2  *  *  *  root    test -d /mnt/mirror/iso && /usr/bin/rsync -avz --stats --delete --progress /disk/iso/ /mnt/mirror/iso/ 2>&1|tee -a /var/log/rsync.log|mutt -s "rsync-backup $HOSTNAME" root@faix.cz

Rdiff-backup

Remote computers do differential backups using rdiff-backup tool.

apt-get install rdiff-backup

For more about Rdiff-backup configuration you can read here.

Lighttpd and Collect daemon monitoring

I liked Lighttpd before Nginx become famous and widely used. It serves me for redirecting http(s) requests for my faix.cz domain.

It also hosts my simple management server web page which is used as a list of all LAN web services.

Collectd monitoring web server is used for usage statistic of my home-server and home-wifi devices.

/etc/lighttpd/lighttpd.conf
/var/www/index.html
/etc/collectd/collectd.conf

/var/www/collection3/

chown -R www-data:www-data /var/www/*

For more about Lighttpd and Collectd configuration you can read here.

Fail2ban

apt-get install fail2ban

To protect myself from bot scanning zombies and script kiddies I love to use Fail2ban service. It protect my SSH, Postfix (SASL) and Lighttpd checking the failed logins and banning annoying zombies for one year based on their IP addresses.

/etc/fail2ban/jail.conf

Logwatch

apt-get install logwatch

Get your daily readers digest from your server logs. It may give you some hints if something is going wrong.

/usr/share/logwatch/default.conf/logwatch.conf

Detail = High

mkdir /var/cache/logwatch

OSSEC

Using HIDS (Host-based Intrusion Detection System) is a good way to monitor and analyze your server logs and get notifications immediately in case something goes wrong.

For more about OSSEC installation and configuration you can read here.

Transmission-daemon

apt-get install transmission-daemon

/etc/transmission-daemon/settings.json
/etc/default/transmission-daemon

For more about Transmission daemon Web interface installation and configuration you can read here.

Monit client

apt-get install monit install Monit client

/etc/monit/conf.d/home-server

For more about Transmission daemon Web interface installation and configuration you can read here.

Ntopng

Setup Ntop network traffic monitor.

For more about Ntopng installation and configuration you can read here.

LinuxCounter

Join more than one hundred thousands Linux registered users and their machines.

For more about LinuxCounter registration, installation and configuration you can read here.

DynDns

In case you don’t have static public IP address from your ISP, use DynDns service and grab some easy to remember domain name for your Linux machine.

For more about DynDns registration, installation and configuration you can read here.

Random generator

apt-get install rng-tools
/etc/default/rng-tools

HRNGDEVICE=/dev/urandom

Sensors

sensors-detect

SmartmonTools

/etc/default/smartmontools

start_smartd=yes

HDD temperature

/etc/default/hddtemp

RUN_DAEMON="true"

Safe LUKS headers

If you use LUKS encrypted drives and you ever lost your header, all your encrypted data get lost forever.
Keep your LUKS header backup file and it may save your sore ass.

cryptsetup luksHeaderBackup --header-backup-file luks_header_disk-backup /dev/mapper/vg0-lvDisk

Backup configuration on every Debian installation

Články , , , ,

The daily cron job does little bit more, actually.

  • create gzipped tar with /etc and other named directories
  • it creates list of extra installed packages from repositories
  • copy backup over LAN or Internet, checks if NFS disk is mounted or remote directory exists
  • it uses tar, cp and rsync commands

You can add it to cron via this commands:

touch /etc/cron.daily/backup-configuration
chown root:root /etc/cron.daily/backup-configuration
chmod 755 /etc/cron.daily/backup-configuration

/etc/cron.daily/backup-configuration

#!/bin/bash

## global variables

# change variables according to your needs

# destination remote directory to save backups
remote_directory=/disk/backup/home-desktop
# destination local NFS mount point
local_mount_point=/disk
# source local directory to store backups
local_directory=/var/backups
# destination SSH login and server name to save backups
remote_server_login=jan@faix.cz
# list of configuration directories and files you want to backup
backup_list="/etc \
             /usr/local/bin \
             /usr/local/etc \
             /var/spool/cron \
             /var/backups/*.0 \
             /var/backups/*.bak
             /var/backups/*.txt"
# parameter the script was launched with or set default LAN
if [ -z "$1" ]; then
    parameter=-LAN
  else
    parameter="$1"
fi

# do not change following variables
# local server name
machine_name=`hostname -s`

## functions
bck_cfg() {
  aptitude -F %p search "?and(?installed,?not(?automatic))" > $local_directory/pkglist.txt
  tar -czPf $local_directory/$machine_name.tar.gz $backup_list
}
start_job() { printf "\nStarting configuration backup job at `date +%c` ...\n"
}
job_complete() { printf "\nConfiguration backup job successfuly completed on host `hostname` at `date +%c`.\n"
}
dir_not() { printf "\nConfiguration backup job failed on host `hostname` at `date +%c`, $dir_loc does not exist.\n"
}

# check if local_directory does not exist, print error message and exit
if [ ! -d $local_directory ]; then
  dir_loc="local directory $local_directory"
  dir_not && exit
fi

# in case the parameter -LAN or -Internet is used, start configuration backup
case $parameter in
    -LAN)
    ## configuration backup job inside LAN
    dir_loc="local directory $local_directory"
    if [ ! -d $remote_directory ]; then
        mount $local_mount_point
    fi

    if [ -d $remote_directory ]; then
        start_job && bck_cfg
        cp $local_directory/$machine_name.tar.gz $remote_directory
        job_complete
      else
        dir_not
    fi
    ;;
    -Internet)
    ## configuration backup job over Internet
    dir_loc="remote directory $remote_directory"
    if (ssh $remote_server_login "[ ! -d $remote_directory ]"); then
        dir_not_ && exit
    fi

    if [ -d $local_directory ]; then
        start_job && bck_cfg
        rsync -azs --stats $local_directory/$machine_name.tar.gz $remote_server_login:$remote_directory/
        job_complete
      else
        dir_not
    fi
    ;;
    *)
    ## print help
    printf "\nThis script create machine configuration backup and copy it over network to destination server.\n"
    printf "\nConfiguration backup script must be run with one of the floowing parameter:\n"
    printf "\n\t-LAN\t\tbackup configuration within LAN to mounted NFS disk"
    printf "\n\t-Internet\tbackup configuration over Internet via rsync to destination folder\n"
    printf "\t\t\tDon't forget to exchange SSH keys running \"ssh-copy-id $remote_server_login\" from source machine you launch backup configuration script.\n"
    printf "\nCheck variables defined in begining of this script:\n"
    printf "\n\tremote_directory=$remote_directory"
    printf "\n\tlocal_mount_point=$local_mount_point"
    printf "\n\tlocal_directory=$local_directory"
    printf "\n\tremote_server_login=$remote_server_login"
    printf "\n\tbackup_list=$backup_list"
    printf "\n\tmachine_name=$machine_name"
    printf "\n\tdefault parameter=$parameter"
    printf "\n\n"
    ;;
esac

2024 Webové stránky Jana Faixe, res publicae. Powered by WordPress and Bootpress Theme