KPTree Home Server Setup

Home Server Web Notes Summary

The main reason for these notes is a reference to assist me with maintaining my home server. This includes upgrading the existing or setting up a new server in the future.

There are many reasons to setup a home server and many different options available. For me one of the big reasons is the tinkering and learning associated with such a set up. There are many other benefits. Perhaps also one of the largest negatives is also the time invested in this endeavour, it will certainly not be for everyone!

I have published these notes on my public website KPTree.net, for my own access and also and possible benefit to others. At this time I am not interest in adding advertising to this site. As these are my personal notes, provided without cost, I assume no obligations in anyway should anyone in anyway use them in full or part. YOU USE THESE NOTES AT YOUR OWN RISK!

I have used many references from the Internet to assist me with the development of my home server and these notes. In general these references links are provided in the relevant section of the notes. Many of these reference links are also provided in the KPTree-Miscellaneous Links. The biggest single source of information and arguably inspiration has come from Havetheknowhow.com, this is certainly a good starting point if you are interested in a Linux based home server!

My Home IT Setup

Hardware - I have censored this for the time being....

A special mention goes to the OpenSprinkler sprinkler controller, that is probably the best network interfaced sprinkler controller available, both for home or commercial use.

Another special mention is Snapraid. I believe this to be the best solution for a modern home server solution, giving the best compromise between performance, reliability and power saving. It should be considered that traditional full time raid systems require all harddisks spinning when in use, compromising long term reliably of all the included disks and increased power consumption. A key benefit of many traditional raid systems, is increase bandwidth (speed) due to use of simultaneous disks, however a modern 3.5" harddisk has a data bandwidth similar to a 1Gb/s Ethernet, so the tradition raid speed benefits are of little value unless a more exotic network arrangement is used. I use an SSD for my main system drive and 2x 6TB hard disks for main datastore + 1 extra 6TB HD for a parity harddisk with snapraid. All the 6TB hardisks are programmed to spin down after 20 minute of no access use. Further to this I back up the 2x 6TB HD to external drives intermittently and have addition 2.5" portable drive with regularly used data and irreplaceable personal data. Some photos, the main irreplaceable data are with other family members, giving some limited effective offsite data backup. I should consider off site backup of the irreplaceable data; to be sure, to be sure.

Network Setup

The home server I have has 4 Intel Gigabit NICs. The past couple of years I have only been using 1 NIC to a main 24 port gigabit switch. This is described in the Basic Network Setup below. The home server has 4 drive, 1 SSD system drive, 2 larger data storage drives and 1 drive used as a parity drive for off line raid for the data storage drives. For most the time a single NIC will provide sufficient bandwidth between the server and switch. However there exists server capacity to saturate bandwidth of a single gigabit NIC. To increase effective bandwidth there is an option to bond 2 or more NIC together to combine their bandwidth. This is call NIC Bonding. To allow virtual machine NIC access the NIC(s) must be setup in bridge mode. Furthermore bridging NICs can also allow the NICs to act as a switch, obviously where more than one NIC is available. The Full Network Setup section below, describes seting up the system with bonded and bridge NICs. Both setups were found to operate well.

Some references are noted below Network Setup Links.

Basic Network Setup

To check available interfaces and names: "ip link"

Ensure the bridge utilites are loaded: "sudo apt install bridge-utils"

Edit the network configuration file: "/etc/network/interfaces" as follows:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# auto eth0
# iface eth0 inet dhcp

#Basic bridge setup on a NIC to allow virtual machine NIC access
#The DHCP server is used to assign a fixed IP address based upon MAC
auto br0
iface br0 inet dhcp
        bridge_ports eth0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

#No point enabling NIC that are not being used
#auto eth1
#iface eth1 inet manual

#auto eth2
#iface eth2 inet manual

#auto eth3
#iface eth3 inet manual

I tried earlier to use static assigned IP setup, but had problems with operation and used setup with dhcp, which worked. I then setup the dhcp sever to assign a fix IP address to the eth0 address.

Full Network Setup

As noted in the main section I have a server with 4 built in Intel NICs. To reduce performance reduction due to limited Ethernet bandwidth using only one NIC I propose to use 2 NICs in bonded configuration and also use bridging to allow server virtual machine access to the NICs and also use the remaining 2 NICs effectivily as a switch.

To check available interfaces and names: "ip link"

Ensure the bridge utilites are loaded: "sudo apt install bridge-utils"

The bonded configuration needs ifenslave utility loaded: "sudo apt install ifenslave"

My NIC connectors are setup as follows:

 IPMI_LAN
 USB2-1   USB3-1   LAN3(eth2)   LAN4(eth3)
 USB2-0   USB3-0   LAN1(eth0)   LAN2(eth1)   VGA

Edit the network configuration file: "/etc/network/interfaces" as follows:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5)
# and brctl(8).
# The loopback network interface
auto lo
iface lo inet loopback

#Setup the Bond
auto bond0
iface bond0 inet manual
post-up ifenslave bond0 eth0 eth1
pre-down ifenslave -d bond0 eth0 eth1
bond-slaves none
bond-mode 4
bond-miimon 100
bond-downdelay 0
bond-updelay 0
bond-lacp-rate fast
bond-xmit_hash_policy layer2+3
#bond-mode 4 requires that the connected switch has matching
#configuration

#Start Bond network interfaces in manual
auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

#Setup Bridge Interface
auto br0
iface br0 inet static
address 192.168.1.5
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255
gateway 192.168.1.1
dns-nameservers 192.168.1.1
bridge_ports bond0 eth2 eth3
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

The following is a description of some of the parameters

  • Bonding
    • bond-mode
      • balance-rr or 0 (default) is a good general option
      • 802.3ad or 4 requires a switch that is corespondingly setup with IEEE 802.3ad Dynamic link aggregation.
    • bond-lacp-rate, only required for 802.3ad mode, Option specifying the rate in which we'll ask our link partner to transmit LACPDU packets, default is slow or 0:
      • slow or 0, Request partner to transmit LACPDUs every 30 seconds
      • fast or 1, Request partner to transmit LACPDUs every 1 second
    • bond-xmit_hash_policy
      • layer2 (default)
      • layer2+3
      • layer3+4
    • layer2 and layer2+3 options are 802.3ad compliant, layer3+4 is not fully compliant and may cause problems on some equipment/configurations.

    • bond-slaves
    • bond-master

Some helpful commands and comments:

  • The NetworkManager is not required on a server, as the base ifconfig and related commands provide full functionality. NetworkManager may conflict with base configuration. Remove with "sudo apt remove NetworkManager". (To see information on system network start-up and ongoing status: "sudo systemctl status NetworkManager" or more comprehensivily "journalctl -u NetworkManager")
  • To see bridge status information: "brctl show"
  • To see bond setup status: "cat /proc/net/bonding/bond0"
  • To list network configuration: "ifconfig", "ip a", "ip route"
  • Kernal IP routing table: "route"

Setup NTP server

The NTP server setup is quite simple, I used the reference from Setting up NTP on Ubuntu 14.04. I replace the pool servers with my local ones, "sudo vim /etc/ntp.conf".

Some NTP tips:

  • To install: "sudo apt install ntp ntpdate"
  • To edit main configuration file: "sudo nano /etc/ntp.conf"
  • The NTP Pool Time Servers page provides the regional NTP servers/pools
  • The systemctl command (status, stop, start, restart): "systemctl status ntp.service"

Setup to Administer Ubuntu Server using VNC

The basic set up is given in have the know how Ubuntu Sever install VNC, with more detailed startup details given in Ubuntu Server: How to run VNC on startup

I prefer a full xfce desktop to a cut down gnome one, so I installed it instead, see How to Install and Configure VNC on Ubuntu 16.04 from Digitalocean.

I basically follow the Have the know how instructions, but instead of "sudo apt install gnome-core", use "sudo apt install xfce4 xfce4-goodies". I have been using vnc4server, not tightvncserver. Also in ~/.vnc/xstartup, only:

  • #!/bin/bash
  • unset SESSION_MANAGER
  • startxfce4 &

(Basically the startxfce4 &, instead of metacity &, gnome-settings-daemon &, gnome-panel &)

The xfce screen-saver seems to default on and use significant system resources, and is basically unnecessary on a headless server. To disable perform the following:

  • In the xfce desktop go to the "Applications Menu > Settings > Screensaver" and disabled the screensaver, then from the "File" menu chose "Kill Daemon".
  • Then go to the "Applications Menu > Settings > Session and Startup" and un-checked "Screensaver (launch screensaver and locker program)" in the "Application Autostart" tab.

(The xfce screensavers actually look quite nice, and may make sense on a standard desktop install.)

The xfce default shell seems to be sh (/bin/sh), I prefer bash (/bin/bash). To check the current shell, type: 'echo $SHELL". To use bash simply type "bash". To make permanent add the line "exec /bin/bash" to the end of "vim ~/.profile". You will need to restart VNCserver for this to take effect.


Some other important tips:

  • To start server "vncserver -geometry 2200x1340". (I have 2 preferred geometries, one for smaller screens (1880x1040) and one for larger (2200x1340))
  • To stop server "vncserver -kill :1" or :2
  • The server log files are stored in ~/.vnc, "less ~/.vnc/KPTreeServer:1.log" or :2. (A log files may contain a number of errors and warnings, however this does not necessarily mean the vncserver will not operate correctly.)
  • The .pid files in ~/.vnc generally show which vnc are currently running, performance can be checked by viewing the log file. The running vnc server process(es) can also be checked with the command "ps -A | grep vnc"
  • The vncserver startup configuration file: "vim ~/.vnc/xstartup"
  • I set up cron to run the following script at boot: "vim ~/Myscripts/StartVNC.sh", StartVNC.sh:
    • #!/bin/sh
    • /usr/bin/vncserver -geometry 1880x1020
    • /usr/bin/vncserver -geometry 2350x1335
  • The cron script statement is "/home/homedirectory/Myscripts/StartVNC.sh >/dev/null 2>&1"
  • (I elected not to use the systemd setup described in the Digitalocean set instructions as I normally run 2 vncservers with different geometries to allow better performance on tablet/laptop/desktop computers.)

Some preferred graphical programs:

  • Synaptic package manager - a graphical package manager; package: synaptic (To use: "gksudo synaptic &", will only work with root privileges.)
  • Virtual machine manager - a graphical virtual machine manager; package: virt-manager
  • XnView - a photo display manipulation program
  • Byobu - a fancy terminal; package byobu
  • Gnome file manager; package: nautilus. (CLI: "gksudo nautilus &", but be very careful if using in root...)
  • Gnome disk utility; package: gnome-disk-utility. (CLI: gksudo gnome-disk &", but be care if using in root...)
  • Gnome disk usage utility; package: boabab. (CLI: boabab &)

SWAP Files

As I have a computer with enough memory I see no need or value in a SWAP partition. In fact as I am using a SSD for the system drive a SWAP is a concern to the reliability of his drive. The following is a list of method to check and disable SWAP function.

  • The command "sudo swapoff -a" turns of an existing SWAP partition. ("sudo swapon -a" turns it back on)
  • To prevent a SWAP partition being mounted at boot comment out the swap partition in /etc/fstab, "sudo vim /etc/fstab". (Another open is to instead use the swapoff -a command in a boot cron job. This allows the swapon -a option to be later used.)
  • The command "free -hw" shows the current memory status.
  • Some links:
    • How do I disable swap? This article also refers to another that explains why turning of SWAP, even with a lot of RAM, may not be best, however this was written at a time SSD were not common and system RAM availability were in general significantly lower.
    • How To Add Swap Space on Ubuntu 16.04. It is interesting that this article also warns against use of SWAP partitions with SSD storage. This article also mentions the swappiness and vfs_cache_pressure setting.

See How To Add Swap Space on Ubuntu 16.04 section on cache pressure on how to adjust this parameter.

NUT (Network UPS Tools) Setup

NUT is an open source network monitoring and control system. I use a Powershield Commander RT PSCRT1100 a 1100VA/880W, line interactive sinewave UPS. It turns out that this is a slightly modified version of the Voltronic Power Otima II 1.1K. Presumably the unit has been slightly modified for Powershield for the Australian market, mainly hardware plugs and presumably some firmware/software setup. In fact Voltronic Power, headquartered in Taiwan, even states they are a leading OEM/ODM manurfacturer of UPSs, AVRs and inverters. The software provided with the UPS is both propriety and bloated. I want a simple efficient command line / daemon based utility that can monitor performance, control as necessary and report on abnormal operation. Unfortunately the NUT project documentation is not simple to follow. I set this up quite some time ago, so this will not necessarily be up to date.

It turns out the nutdrv_qx driver is usually better suited for Voltronic Power UPSs than the Blazer_usb one. See NUT Voltronic Power UPS Protocol, it is also list under the Voltronic Power section, (various)USB of the NUT Hardware compatibility list

NUT Configuration Files

NUT can be installed by "sudo apt install nut". The configuration files are located at /etc/nut :

  • nut.conf
    • MODE=standalone
  • ups.conf
    • [KPBUPS]
    • # driver = blazer_usb # works
    • driver = nutdrv_qx # works with more info than blazer_usb
    • port = auto
    • desc = "KPTreeServer UPS"
    • default.battery.voltage.high=26.9
    • default.battery.voltage.low=21.5
    • default.battery.charge.low=25
    • default.battery.charge.warning=60
    • default.battery.charge.restart=30
    • default.battery.runtime.low=420
    • default.battery.runtime.restart=180
  • upsd.conf
    • LISTEN 192.168.x.x #IP address of NUT server
    • LISTEN 127.0.0.1 3493 #IPv4 NUT port
    • LISTEN ::1 3493 # IPv6 NUT port
  • upsd.users
    • [admin]
    • password = {password}
    • actions = SET
    • instcmds = ALL
  • upsmon.conf
    • MONITOR KPBUPS@localhost 1 admin {password} master
    • MINSUPPLIES 1
    • SHUTDOWNCMD "/sbin/shutdown -h +0"
    • NOTIFYCMD /home/baumkp/Myscripts/nut-notify.sh
    • POLLFREQ 5
    • POLLFREQALERT 5
    • HOSTSYNC 15
    • DEADTIME 15
    • POWERDOWNFLAG /etc/killpower
    • NOTIFYFLAG ONLINE SYSLOG+WALL+EXEC
    • NOTIFYFLAG ONBATT SYSLOG+WALL+EXEC
    • NOTIFYFLAG LOWBATT SYSLOG+WALL+EXEC
    • NOTIFYFLAG FSD SYSLOG+WALL+EXEC
    • NOTIFYFLAG COMMOK SYSLOG+WALL+EXEC
    • NOTIFYFLAG COMMBAD SYSLOG+WALL+EXEC
    • NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
    • NOTIFYFLAG REPLBATT SYSLOG+WALL+EXEC
    • NOTIFYFLAG NOCOMM SYSLOG+WALL+EXEC
    • RBWARNTIME 43200
    • NOCOMMWARNTIME 300
    • FINALDELAY 5
  • upssched.conf
    • CMDSCRIPT /bin/upssched-cmd # default

Ensure ownership and permissions are as follows: "sudo chown root:nut /etc/nut -R" and "sudo chmod 640 /etc/nut -R"

NUT Output

The output of "/bin/upsc KPBUPS@localhost", (or simply upsc KPBUPS):

Init SSL without certificate database
battery.charge: 100                  #calculated value
battery.charge.low: 25               #Set value LB (Low Battery Alarm)
battery.charge.restart: 30           #Set value
battery.charge.warning: 60           #Set value
battery.energysave: no
battery.packs: 1                     #Fix value
battery.protection: yes
battery.runtime: 1680                #calculated value
battery.runtime.low: 420             #Set value
battery.runtime.restart: 180         #Set value
battery.voltage: 27.30
battery.voltage.high: 26.9           #Set value for calculation
battery.voltage.low: 21.5            #Set value for calculation
battery.voltage.nominal: 24.0        #nominal/design value
device.model: LIHVX1K1               #Fix value
device.type: ups                     #Fix value
driver.name: nutdrv_qx
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: auto
driver.version: 2.7.2                #Fix value
driver.version.data: Voltronic 0.01  #Fix value
driver.version.internal: 0.06        #Fix value
input.current.nominal: 4.0           #nominal/design value
input.frequency: 50.0
input.frequency.nominal: 50.0        #nominal/design value
input.phases: 1                      #nominal/design value
input.voltage: 241.7
input.voltage.nominal: 230.0         #nominal/set value
outlet.0.switchable: no
output.current: 0.9
output.current.nominal: 4            #nominal/design value
output.frequency: 50.0
output.frequency.nominal: 50.0       #nominal/design value
output.phases: 1                     #nominal/design value
output.power.maximum.percent: 22
output.power.minimum.percent: 21
output.powerfactor: 0.8              #nominal/design value
output.voltage: 241.6
output.voltage.nominal: 230.0        #nominal/set value
ups.beeper.status: enabled
ups.delay.shutdown: 30
ups.delay.start: 180
ups.firmware: 00303.05               #Fix value
ups.firmware.aux: P00                #Fix value
ups.load: 21
ups.power.nominal: 1100              #nominal/design value
ups.productid: 5161                  #Fix value
ups.start.auto: yes
ups.start.battery: yes
ups.status: OL
ups.temperature: 19.4
ups.type: line-interactive           #nominal/design value
ups.vendorid: 0665                   #Fix value

NUT & Systemd

I have been using NUT on my server and UPS now for about 4 years. In all that time I never got the upsmon part of it operational. This monitors the UPS and sends out messages and shutdown the server on UPS low battery.

The current NUT scripts seem to be based upon init.d (located in /etc/init.d), in particular /etc/init.d/nut-server and /etc/init.d/nut-client, (with /etc/init.d/nut-monitor being a link to /etc/init.d/nut-client). Systemd seems to have a method for legacy init.d scripts and creates scripts in /run/systemd/generator.late, in particular, ups-monitor.service.

To see running processes use: "sudo ps -e | grep nut*" and "sudo ps -e | grep ups*"

Turn off the existing legacy scripts and systemd handlers, as noted above; "sudo update-rc.d nut-client disable", "sudo update-rc.d nut-server disable" and "sudo systemctl disable ups-monitor.service"

The Kepstin blog gives a good NUT and Systemd discussion Network UPS Tools (nut) and systemd. However it is written for Fedora and there are some nuanaces required to work with Ubuntu. For a general understanding of Systemd see DigitalOcean references: Understanding Systemd Units and Unit Files, How To Use Systemctl to Manage Systemd Services and Units and Systemd Essentials: Working with Services, Units, and the Journal.

First we need to get the NUT driver running, "sudo vim /etc/systemd/system/nut-driver.service"

[Unit]
Description=Network UPS Tools - power device driver controller
After=network-online.target

[Service]
ExecStart=/sbin/upsdrvctl start
ExecStop=/sbin/upsdrvctl stop
Type=forking

[Install]
WantedBy=multi-user.target

My system takes a bit of time for the network system to fire up, so I delay the NUT driver startup until after the network is up, by adding the directive "After=network-online.target".

Unfortunately when this script is run, "sudo systemctl start nut-driver.service" it reports failure as it is unable to access the directory "/var/run/nut". So we can create the directory with: "sudo mkdir /var/run/nut sudo chown root:nut /var/run/nut sudo chmod 770 /var/run/nut". This allows nut-driver.service to run, but fails after a reboot. Further investigation shows that "/var/run" is a symlink to "/run" and the command "df -T" shows that /run is a temporary file system mounted in RAM that must be recreated every boot. Inspection of the init.d NUT files show they include creation of the /var/run/nut directory. In systemd .conf files must be created in the /usr/lib/tmpfiles.d that create these emphemeral directories, see "man tmpfiles.d". So we need to create the file: "sudo vim /usr/lib/tmpfiles.d/nut-driver.conf" and populate so:

#Type Path        Mode UID  GID  Age Argument
 d    /run/nut    0770 root nut  -   -

Next in order to ensure correct handling of emergency poweroffs and test reboots correctly, check the /lib/systemd/system-shutdown/nutshutdown file exists; "sudo vim /lib/systemd/system-shutdown/nutshutdown" and populate so:

#!/bin/sh
/sbin/upsmon -K >/dev/null 2>&1 && /sbin/upsdrvctl shutdown

Next create the systemd nut server service for /sbin/upsd, configuration file, "sudo vim /etc/systemd/system/nut-server.service

[Unit]
Description=Network UPS Tools - power devices information server
After=nut-driver.service

[Service]
ExecStart=/sbin/upsd
Type=forking

[Install]
WantedBy=multi-user.target

Next create the systemd nut monitor service, /sbin/upsmon, configuration file, "sudo vim /etc/systemd/system/nut-monitor.service

[Unit]
Description=Network UPS Tools - power device monitor and shutdown controller
After=nut-server.service

[Service]
ExecStart=/sbin/upsmon
PIDFile=/run/nut/upsmon.pid
Type=forking

[Install]
WantedBy=multi-user.target
NUT Key commands and scripts
  • List all the UPS device profile variable "/bin/upsc KPBUPS@localhost 2>/dev/null"
  • A script file to succinctly log the UPS data, using cron to run script every 10 minutes "sudo Myscripts/UPSScan.sh 2>/dev/null"
  • A script file to rotate a log file out to a dated gzip log file, with a new empty log file, the log file with path is the script input "sudo rotatelog.sh /var/log/UPSLog.Log", again can be use with cron monthly to rotate a log file.
  • The NUT service (start / stop / status / restart) command:"sudo systemctl status nut-server.service"
  • To list the available UPS commands: "upscmd -l KPBUPS"

The NUT binaries issue a SSL error to STDERR, so the addition of 2>/dev/null suppresses this.

The parameters set in ups.conf; default.battery.voltage.high and default.battery.voltage.low are used to calculate battery runtime and battery charge level and for low battery alarming. These values can be configured to account for varying battery performance, due mainly to type, age.

VIM Customisation

I use the VI (or VIM) editor. It comes standard on most Linux and UNIX distributions, or can otherwise be installed. A key feature I configure is the VIM colour scheme, as the standard colour scheme does not work well with black background terminal windows I prefer to use. Simply create the file on home directory, ".vimrc" ("sudo vim .vimrc") and add the line ":colorscheme desert".

The different VIM colour scheme definition files are located at "/usr/share/vim/vim74/colors"


BASH Customisation

The standard BASH colour configuration uses a blue colour for listing directories (ls) which is difficult to read on a black background. While this is the "standard colour", due to the impracticality I have decided to change it.

The personal BASH user configuration file is: "~/.bashrc". Simply add the following line to this file: "LS_COLORS='di=1;32' ; export LS_COLORS" The code 1;32; is for a light green colour.

The .bashrc file also has a number of other interesting "features" and options, such as aliases and colour prompts. If you turn on the colour prompt option (force_color_prompt=yes), again the dark blue colour may be difficult to read so I change the prompt color code from 34 to 36.


BASH History Customisation and Use

How To Use Bash History Commands and Expansions on a Linux VPS

VIM Text Editor

A powerful text editor, standard in most Linux distributions and available in Windows. Need some time and effort to learn though, particularly if moving from graphical user environment.

Some Informative Links:

Some Quick tips:

  • There are two (2) main modes: Command mode and Insertion mode. You only normally type text in Insertion mode. The Esc (escape) key enters command mode and the i or INS(insert) keys return to Insertion mode.
  • If like me you use a keyboard without an insert key, eg. Microsoft Surface, you can get into insert mode directly from command mode by typing i. When you open VIM you are in command mode, so simply type i (or insert) to get into insert mode.
  • To copy, cut and paste:
    • First go into command mode (ESC or CTRL-[)
    • Move using cursor keys to place to start highlight, hit v key and highlight area to be copied (or cut)
    • key y to copy, or d to cut
    • Move to place to paste, key P to paste before cursor or p to paste after

Rsync - File sycronisation, full feature file copy

There is a lot of existing information on using Rsync published

fsck - file system check

Use fsck to check and repair a file system. The file system must be unmounted when being check and repaired to prevent corruption!

Checking Root File System

The root file system can not be unmounted and checked. Two possible options to check the system are:

  • Force the Root Filesystem to use fsck during reboot, The Geek Stuff covers this well in, How to Force Check Root Filesystem using fsck During Reboot. A summary of key commands:
    • The Root file system is automatically checked at certain intervals. To check this: "sudo tune2fs -l /dev/sda1 | grep -i check".
    • To force a recheck at the next reboot, create the file forcefsck. At the next reboot a fsck will be performed and the forcefsck file removed. Commands:
      • "cd /"
      • "sudo touch forcefsck"
  • Boot the system to a rescue file system. From the rescue file system the main boot drive may then be checked with fsck. (Further details not given.)

KVM Guest Corrupted - Recovery Options and Related

Some Keypoints are:

  • "sudo virsh" to get into virsh, the virtualisation interactive terminal. Once inside virsh:
    • "list" to list running VMs, or "list --all" to list all defined VMs, running or shutdown
    • "edit VM_name" to edit the XML configuration file of the VM names VM_name.
    • "sudo virsh define XXX.xml" to add a vm (XXX.xml)into virsh persistently. The VM is not started. The vm xml definition files can be found in "/etc/libvirt/qemu".
    • "sudo virsh start VM_name" to start the VM. (Also reboot, reset, shutdown, destroy)
  • "sudo virsh help | less" list all the virsh commands

Some other file system tips:

  • To check current mounted disks use "df"
  • To check current disks use "sudo fdisk -l". (This is a disk partition command. Be careful with use.)
  • To check current disks use "sudo parted -l". (This is a disk partition command, more modern than fdisk. Be careful with use.)
  • To check directory size use "sudo du --max-depth=1 --all"
  • The filesystem boot file is /etc/fstab, "sudo vim /etc/fstab". (Be careful with editing this.)

Other Important Service to Setup

FailtoBan

Scans log files and check for in appropriate password activities and update and uses firewall (IPTables) to restrict (stop for a period of time) these activities. So failtoban limits incorrect authorisation attempts, thereby reducing, but not entirely eliminating associated risks and bandwidths. It is primarily used on port and associated services open to the public. DigitalOcean How To Protect an Apache Server with Fail2Ban on Ubuntu 14.04 and How Fail2Ban Works to Protect Services on a Linux Server. Also see the wiki of Fail2Ban on nftables and Fail2ban Add support for nftables #1118 and Add nftables actions #1292.

MONIT

Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. The email server instructions from Ex Ratione - A Mailserver on Ubuntu 16.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube also include some installation instructions for monit.

Another site with some security tips, How to secure an Ubuntu 16.04 LTS server - Part 1 The Basics

Tripwire

Trip wire check system files to check for any changes and alarms / alerts upon changes.

Set Up and Ubuntu APT Cache

The apt-cacher-ng looks to be a self container apt caching server. Basically the apt cacher stores all the relevant apt update and upgrade related files and and acts as a proxy server to multiple clients. A handy feature to improve speed and reduce Internet bandwidth where a virtual machine server is used with multiple clients. There is another package called apt-cacher but it depends upon the installation of a separate webserver.

There is also APT-mirror that retrieves all packages from the specified public repository(s). Where as apt-cacher only retrieves each package when called and stores for subsequent use by other clients. APT caching looks the way to go and apt-cacher-ng the best overall option. I installed apt-cacher-ng on the VM server, not a VM client. The clients are setup to obtain their apt updates and upgrades via the server.

The LinuxHelp web page, How To Set up an Apt-Cache Server using "Apt-Cacher-NG" in Ubuntu 16.04 Server, provide a good description of how to setup. It is reasonably straight forward. I suggest the use of "sudo systemctl restart apt-cacher-ng", as opposed to the old fashion "sudo /etc/init.d/apt-cacher-ng restart".

If the non-default Cache directory is not set up correctly the program defaults to "/var/cache/apt-cacher-ng". This quirk is covered in How to change the directory of the apt-cacher-ng downloaded packages" in Ubuntu Xenial.

Links to the Apt-Cacher NG home page and Apt-Cacher-NG User Manual.


To access apt-cacher-ng web page: "http://192.168.1.5:3142"

There is an issue with use of apt-cacher and SSL/TLS repositories. A good reference is from packagecloud:blog: Using apt-cacher-ng with SSL/TLS.

The following is a list of related commonly used commands and scripts:

  • Systemctl related commands:
    • Systemd common commands (start / stop / restart / status) (enable / disable for boot control)
    • List current Systemd operating units: "sudo systemctl list-units | grep '*'". Change or remove the grep statement as required.
    • Check the boot journal: "sudo journalctl -xe"
  • To check running process with open for openvpn(/del, for deluge) "ps -A | grep open"
  • To change time zone from command line: "sudo dpkg-reconfigure tzdata".

Some related links

Ubuntu Network Setup Links

Links relating to bridged and bonded Networking

A bridged network allows different networks to be connected, both physical, like NICs or Wifi and virtual, allowing virtual machine to connect to a physical network and even be assigned a LAN IP address. Bonding allows physical networking devices such as NICs or Wifi to be bonded to allow increased bandwidth or redundancy. Sadly there seems to be alot of information out there that isceither for older version of software or other purposing.