Virtualmin.. Holy Shit!

If you already know what VPS and Virtualmin are, you will probably want to skip straight to the tips. If not, do read on.. Virtualmin logo, in transparent webp

Introduction..

Okay, so it's been a while since I did any sort of web server setup for the org here. I was mostly content to mess around with Plesk and Cpanel hosting; other less-than-optimal solution. But the truth is, they suck.

In a conversation with my recent host's tech support staff, in answer to my question,

"Why can't I switch that sh*t the f*ck off?"

He says,

"You need a VPS"

I think, "a what?", which just just goes to show how out of the web hosting loop I have become. My previous host (hope you are okay man!) would just flick the switch. I'm reminiscing... So.. VPS. He was right. It was exactly what I needed. So at least, thanks for that.

Virtual Private Server

It's like your own dedicated server, except virtual, so the host can slam a few on a box and if all goes to plan, each server has dedicated, guaranteed resources, very much like a dedicated server does.

If you want to have total control over your hosting environment, a VPS, or dedicated server, is for you. With every single hosting company I've used, I've had to make compromises. No more! With a VPS, everything is under your control.

A dedicated server is expensive. A VPS is cheap. Surprisingly so. The hosting company doesn't need to do anything, you see. On an "unmanaged" server, you basically get a net-connected box and an IP and away you go. You can get decent VPS hosting for way cheaper than shitty shared reseller hosting. Word.

Some VPS companies even do free trials, though I suspect it would be simpler and probably cheaper to just head along to LowEndBox and see what's on offer this month. Or last month. Browse around; you may be amazed by just how cheap VPS hosting can be. I was.

The downside is that as it doesn't even come with an operating system installed, so either a) you pay some company, probably the VPS hosting company, to manage the server for you, which makes it expensive again. Or b) you set it up yourself.

Thing is, setting up a web server is no longer a simple operation, even for advanced Linux users. If you want all the modern, multi-user features one expects from "professional" hosting, it will be a massively complex and tricky operation, taking days or weeks to complete.

Fortunately you don't have to..

Enter Virtualmin

And Webmin, of course. They come as a package. And wow! What a package!

I remember Webmin from days of yore. Handy. This Virtualmin/Webmin thing is a whole other level, literally. You to run as many web sites as your hardware can handle. Think hosting panel on steroids. It basically enables complete control of your box, from the intricacies of domain management, right down to the basic system hardware and software setup. And all from a nice blue, mobile-friendly web interface (yes, there are themes).

Once you have your chosen OS installed (your VPS company will have a control panel where you can select and install an OS, or their tech guys will do it for you), you can ssh into your box and install Virtualmin with one single command, well, two, if you include the download..

wget https://software.virtualmin.com/gpl/scripts/install.sh
/bin/sh install.sh

If you aren't logged in as root, prepend the install command with sudo.

I'm not going to go into details about setting up a VPS, because that has already been done excellently elsewhere.* Most big VPS hosting companies have tutorials, too. Nor am I going to run through Virtualmin setup, for the same reason.

Suffice to say, you run the command, Virtualmin does a heap of installs, your console scrolls and at the end of it all, you have a feature-rich, open source hosting control panel that wipes the floor with commercial offerings.

The first time you open it, your mind will be blown, but after a while you get familiar with its many many tabs and features and as you realise the huge amount of work and thought that went into it, it starts to make sense. Incredible stuff. And all free.

Okay, it's not for absolute beginners, but if you are comfortable enough in a console (and with Virtualmin, you rarely, if ever need to use an actual console** if you don't want to) and don't mind doing a bit of research and fiddling, it's certainly within your capabilities. Whatever happens, you will learn loads.

So, to summarise... VPS + Virtualmin = Hosting Nirvana. Doo eeeet.

* Pay special attention to his advice about using Cloudflare for your DNS. This is a Very Good Idea, for many reasons. And also free. Word on the street is there is practically no limit to the number of domains you can setup on a Cloudflare free account. Their interface is polished and mature and also catches many user errors. Recommended.
* Yes of course there is one right there in Virtualmin, at the bottom of the navigation panel, the wee >_ icon. THOUGH NOTE: This is NOT an "interactive" shell, so you can't do stuff that requires ncurses, etc., e.g. top/htop. It's basically the same as "Command Shell" module. Useful, but you would be better in a real console.

Virtualmin Tips

Google will provide most of the answers you seek. The Virtualmin "community" is pretty large. Here are some things you might not find easily, or at all, but which might prove useful..

Remote Backups

One of the first things you need to setup, is backups. Ideally daily, and off-site.

Virtualmin offers multiple backup "modules". Each offers a variety of destinations, one being SSH; so if you have another fast server kicking around, or even a Linux rig in your home network, you can easily get scheduled automatic off-site backups. The main Virtualmin backup modules are..

Webmin > Backup Configuration Files

This makes a backup of all your configuration files and the SSH facility works well. You can do strftime substitutions in file names and a destination might look something like this..

/home/USER/VPS_settings_backup_%Y.%m.%d@%I.%M%p.tar.gz

This being the path on the remote server. In this module there is no facility to delete old backups. That is up to you (they aren't even a MB, so no biggie).

The destination file would look something like: VPS_settings_backup_2021.04.08@11.24AM.tar.gz

Curiously, you can ignorantly do $(date +%Y.%m.%d@%I.%M%p) and Virtualmin will strip off the unnecessary "$date()" part for you.

So far so good. However..

Webmin > System > Filesystem Backup

This makes good old file system backups using dump, and the SSH does not work. This backup appears to require a quite specific set of unlikely circumstances; for starters, your remote host needs to have rmt installed. Or some fiddling. Hmm.

At any rate, I gave up on that insecure-looking malarkey fairly quickly and instead used a simple post-backup scp command to achieve the same effect. This one is for my /etc filesystem backup..

scp /backup/etc_fsbackup USER@backup-host.com:/home/USER/etc_fsbackup_$(date +%Y_%m_%d_%I_%M_%p).tar.gz

And that's it. Fully automated filesystem backups to an off-site server.

Hold On! I hear you say. What about the password? Can't be having scripts running in the background asking for passwords!

The answer, of course, is keys*. Login as root to your VPS box, make a key..

ssh-keygen -t rsa -C "USER"

Accept the defaults ("USER" is just a comment, but will help you differentiate the key later - use something that will make the key easy to differentiate in a file of many keys, like "your-name@your-PC-name"), no password (which would defeat the purpose), then copy and paste the new key out of ~/.ssh/id_rsa.pub and into the end of this file..

/home/USER/.ssh/authorized_keys

That's a file named "authorized_keys" (which you may need to create) inside a directory named ".ssh" (which again, you may need to first create) inside your home folder on the remote server, wherever that may be (whichever user you will login as), probably not /home/USER/!

So now there's a copy of your VPS-server-published root user key in the authorized_keys file on the remote host.

Ensure the remote machine's permissions are set to 0700 for ~./ssh folder and 0600 for the ~/.ssh/authorized_keys file (in case "StrictModes" is set in the sshd_config). That's it.

Now the two servers are best friends and your scripts and cron jobs (and you) can happily scp and ssh to the remote server all day long without passwords.

More fun with keys..

I should add that once you have keys installed, it's much easier to (do so many things including) setup the above Filesystem Backups using the built-in SSH facility. The only thing that will likely get in your way is the port allocation.

There is no way, from within Webmin, to change the port used by the built-in tar-over-ssh. And who uses port 22 for SSH these days? The devs on the Webmin forum say it cannot be switched from the default. But it can.

A simple work-around (thanks to dev Jamie Cameron at the Webmin bug tracker)..

Create a /root/.ssh/config file containing :

Host backup-destination.IP.address
  Port 7777

Boom! It works instantly. You would think they would make a button.

NOTE: You can also put host names in this file, so if you had a second VPS, as well as a regular entry, you could fake a nice short entry in /etc/hosts ..

127.0.0.1	localhost localhost.localdomain
::1	localhost localhost.localdomain
REAL-IP-OF-OTHER-VPS  vps2

Then add that fake host to your /root/.ssh/config..

Host vps2
  Port 7777

And then when you need to ssh into your other box you simply do..

ssh vps2

With no need to specify port number or long domain names or hard-to-remember IP addresses. Do the same thing on both boxes.

Sure, you could make an alias in your ~/.bashrc or ~/.bash_aliases files and reduce any length of command to a few simple keystrokes, but this method works everywhere; in scripts, in your shell, wherever ssh or scp ar used.

Virtualmin > Backup and Restore

The backup third option is an entire section enabling backups to Amazon S3, Cloud storage and more. Here you can backup entire virtual servers and their configuration, or restore the same. The Scheduled Backups sub-section is the one most people will probably want, at least initially.

These backups are in a format that is designed for Virtualmin to easily restore, but almost completely useless for humans as the backup is in a zillion numerically-named folders. You will want to setup a schedule for this, of course; in case of need; but for usable backups that you can pluck files from later, the first two backup options (above) are definitely the way to go. So set this, test it, and forget it.

Note: if you are using separate files for each domain (seems smart) what you enter into the "file" input will be used as the backup directory, so no extension required here, simply do..

/home/me/config/config_backup_%Y_%m_%d_%I_%M_%p

And your remote backup will be..

/home/me/config/config_backup_2021_04_05_05_58_PM/MyDomain.com.tar.gz

Or whatever.

With this backup module, put your ssh port directly after the IP address in the usual format: 142.250.187.206:21098

* It goes without saying that you should also be using keys to login to your VPS.

Keys are superior to passwords and switching over leap-frogs a half-dozen half-arsed "accepted" security measures in one stroke, and makes communication between your PC and VPS effortless. Just do it. It literally takes less than a minute. Then disable password access altogether and say goodbye to so-called "hacking" attempts.

If you are on Windows, see here. Once you have created your ssh_private_key.ppk file, you can simply point any program that wants to access your VPS (PuTTY, Kitty, WinSCP, Filezilla, etc.) to that file and never have to think about passwords again.

A quick and obvious note about remote backup security..

It perhaps doesn't go without saying that if you are using a second server for backups, you should NOT be using the root account for these backup operations.

Create a separate user just for backups with permission to only do backups, and create backup directories on the target machine owned by that user. The same username on both VPS seems sensible.

Migrating from Cpanel

This is surprisingly painless. So long as you don't mind having your sites in the same user sub-directory structure as before (and you can change this afterwards).

Virtualmin > Add Servers > Migrate Virtual Server

Then you simply pick your FULL backup file and click. Done.

The downside is how long it might take to upload your full CPanel backup to your new VPS in the first place. We are probably talking many GB. Even if you have super-fast fibre, why tie up your own connexion with this chore? Once again, scp comes to the rescue..

scp -P 21098 USERNAME@premium97.web-hosting.com:/home/USERNAME/backup-4.1.2021_17-03-58_cor.tar.gz /CPanel_Backup.tar.gz

This example copies a backup directly from the old web server (namecheap, where I have been living this last year) to the root of my shiny new VPS. And it happens in seconds instead of far, far longer. So long as you have SSH access to your old server, this is the way.

Note: you will be prompted for the password to your old server. Don't make keys for that server; you won't be back!

Note: I specified the port with "-P 21098", because that's what namecheap uses. If your old server uses the default port (22), you don't need to specify the port.

Note: For obvious reasons, you want to do this before your old hosting plan runs out. If you are migrating from namecheap, this is a surprisingly long time after your plan elapses! (10 days)

Free SSL, for mail, too..

Virtualmin makes it a doddle to get a free SSL certificate from Let's Encrypt. This is another thing hosting companies would prefer you paid for.

What isn't immediately apparent is how easy it is to have this certificate copied over to other servers, like postfix; your mail server. There is, in fact, a button for it!

Virtualmin > SSL Certificates [tab] Service Certificates.

So now you can send and receive mail with a valid SSL certificate. At least for one domain.

Currently there is no working SNI in Virtualmin postfix; it's on it's way mid 2021. In the meantime, when you copy over your certificate, remember THERE CAN BE ONLY ONE! Like Highlander. Whichever certificate you copy over (last!) covers all your domains.

It might be best to use your main domain as a single MX for all your domains until SLI arrives.

Where is my Inbox?

This was a weird one. Logging into Usermin, checking mail, I have no inbox. I migrated from CPanel and have been using Thunderbird; everything works fine, but just wanted to check the webmail was working, as I sometimes need it. Hmm.

Turns out the email was set to use a file rather than a folder (IMAP style). I changed it to folder, restart usermin and bingo! All my emails appear.

Being a few levels deep in the prefs, I of course made a bookmark..

Webmin >> Usermin Configuration >> Usermin Module Configuration >> Read Mail :: Mail storage format for Inbox

Two VPS..

Fairly soon after I got my VPS up and running I realised that what I really needed, was a second VPS.

For around a tenner a year (+VAT!), you can get..

A net-connected root shell box is always fun. Two is more-than-double-funner. Try it.

That "tight, responsive firewall"..

Being Linux underneath, Virtualmin has an excellent firewall. iptables. And the web is stuffed with interesting scripts and tweaks to give you useful firewall features. If you have the time and inclination, that is.

However, if you are a busy type, running a busy internet-facing server, I recommend something that can, in a couple of commands, turn your Linux firewall into the firewall-of-death-to-bad-bots..

ConfigServer Security & Firewall (csf) is, and I quote, "A Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application for Linux servers."

It takes your Linux firewall to the next level. You want to know when processes are running away, or ports are being scanned or login attempts fail and much, much more, this is the firewall appplication you need. The simple instructions even include a section for webmin (basically, install the module and go).

csf can be initially overwhelming, especially the zillion warning emails. But if you take a few minutes with the config file, you will be able to get these and much more under your control. By the way, you can exlclude processes and commands (php cgi, etc) inside /etc/csf/csf.pignore. You can even have it automatically allow dynamic domains. Handy!

csf takes a minute to get under control, but once you do, you won't regret it. Highly recommended.

Miscellaneous Virtualmin tips, tricks and notes for n00bs..

mc: aka. GNU Midnight Commander

If you are new to the console and Linux in general, and even if you aren't, mc is a superbly useful tool for navigating the filesystem and doing stuff. If it isn't installed, you can get it like this (as root, or use sudo)..

#Debian/Ubuntu
apt-get install mc

#Centos/Redhat
yum install mc

Launch mc (by typing mc<Enter> into your console). Hit F9 (Command menu) > Down arrow > Panel Options > Lynx-like motion (enable). OK. Now you can whiz about the filesystem with the arrow keys, like a ninja.

You will probably also want to enable automatic pull-down of menus (when you hit F9). Hit F9 (Command menu) > Down Arrow (for the last time!)> Configuration > Drop down menus (enabled) so you don't have to hit the down arrow to activate the menus in future. Now you are all set.

MC QuickStart:

That should be enough to get you started.

bash

Being Linux, when you login, you get a bash shell. Being a bash shell, it will have a .bashrc file in your home (root) directory, lovingly known as ~/.bashrc . If you are smart, you will edit this file and the associated .bash_aliases, if it exists - yes on Ubuntu, no on CentOS - and pimp your VPS login up to the max, or at least make it useful. A few minutes here could save you hours in the future.

By the way, on CentOS and other Linux that don't use the .bash_aliases convention, you might want to add that, as it's a nice idea to keep your aliases away from your other login commands. Simply add this to your .bashrc..

# Aliases..
if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

Aliases are shortcuts that you create so instead of typing this:

dmesg -T|sed -e 's|\(^.*'`date +%Y`']\)\(.*\)|\x1b[0;34m\1\x1b[0m - \2|g'

You can simply do..

dm

Or whatever. You want to create aliases for any commands that you need to perform often in a shell; restarting servers, tailing log files, whatever you need to do, except more quickly.

For example, this creates an alias named "lports" which displays the currently listening (IPV4) ports..

# current listening ports..
alias lports='lsof -Pni4 | grep LISTEN'

So I can do lports at any time in the console to see the current listening ports. I also have an lports6, which I guess you could string together with a ";".

Over the years, one builds up a collection of login script snippets, from the useful to the downright silly. But once you get a set that works, you will want to replicate those to all your VPS. Yes, they will likely need a wee tweak for different platforms, but not a lot** .

A couple of examples..

If you have csf installed, you could add this..

# Restart ConfigServer Security & Firewall (csf)..
alias ssec='systemctl status csf;systemctl status lfd'
alias rsec='csf -ra;systemctl restart lfd;ssec'

Notice how I used the first alias in the second command.

syslog to have a look at the most recent 200 log lines..

alias syslog='journalctl -b -n 200'

fnd to quickly find with a specified name anywhere in the filesystem..

# QuickFind (TM)
alias fnd="find / -path '/proc' -prune -type f -o -name"

And so on.

CAUTION! Before you add an alias, remember to check their isn't already a system command with that name! Many of the cute two-letter combinations will have already been taken. BE CAREFUL!

** If you have csf installed, there is a useful list of current system binary/log locations at the foot of /etc/csf/csf.conf.

Secure remote backups..

The built-in backup functions are fine for local backups or remote backups to servers you control. But if you want to backup to a server where there is a chance others might get to your backups (like a shared server), you will want to at least use a password protected archive. This is fairly simple to achieve..

Install 7z, if you haven't already..

#Debian/Ubuntu
apt-get install p7zip-full p7zip-rar

#Centos/Redhat
yum install p7zip p7zip-plugins

Then make a script, perhaps /usr/local/bin/secure_backup.sh

#!/bin/bash

# prefs
pass="MyBackupPassword"
#end prefs

# Grab it once NOW and re-use..
filex="$(date +%Y_%m_%d@%I_%M_%p)"

# enter directory to backup.. (I prefer doing it this way)
cd /etc
# create a 7zip archive, max strength, using our password from above.
7z a -mx=9 -mhe -t7z -r /backup/etc_$filex.7z * -p$pass
# transfer to insecure host..
scp  /backup/etc_$filex.7z ME@SomeServer.Com:/home/ME/backup/settings/etc_$filex.7z && rm -rf /backup/etc_$filex.7z

# repeat for other directories..
cd /root
7z a -mx=9 -mhe -t7z -r /backup/root_$filex.7z * -p$pass
scp /backup/root_$filex.7z ME@SomeServer.Com:/home/ME/backup/NY_VPS/homes/root_$filex.7z && rm -rf /backup/root_$filex.7z

For the scp commands to work, you will need to have keys setup. See above. And use your own details, of course.

Welcome to corz.org!

I'm currently setting up a new web server, please bear with me.. See a bug? Mail me!