Virtualmin.. Holy Shit!
Okay, so it's been a while since I did any sort of web server setup for the org here. I was mostly content to mess around with Plesk and Cpanel hosting; other less-than-optimal solution. But the truth is, they suck.
In a conversation with my recent host's tech support staff, in answer to my question,
"Why can't I switch that sh*t the f*ck off?"
"You need a VPS"
I think, "a what?", which just just goes to show how out of the web hosting loop I have become. My previous host (hope you are okay man!) would just flick the switch. I'm reminiscing... So.. VPS. He was right. It was exactly what I needed. So at least, thanks for that.
Virtual Private Server
It's like your own dedicated server, except virtual, so the host can slam a few on a box and if all goes to plan, each server has dedicated, guaranteed resources, very much like a dedicated server does.
If you want to have total control over your hosting environment, a VPS, or dedicated server, is for you. With every single hosting company I've used, I've had to make compromises. No more! With a VPS, everything is under your control.
A dedicated server is expensive. A VPS is cheap. Surprisingly so. The hosting company doesn't need to do anything, you see. On an "unmanaged" server, you basically get a net-connected box and an IP and away you go. You can get decent VPS hosting for way cheaper than shitty shared reseller hosting. Word.
Some VPS companies even do free trials, though I suspect it would be simpler and probably cheaper to just head along to LowEndBox and see what's on offer this month. Or last month. Browse around; you may be amazed by just how cheap VPS hosting can be. I was.
The downside is that as it doesn't even come with an operating system installed, so either a) you pay some company, probably the VPS hosting company, to manage the server for you, which makes it expensive again. Or b) you set it up yourself.
Thing is, setting up a web server is no longer a simple operation, even for advanced Linux users. If you want all the modern, multi-user features one expects from "professional" hosting, it will be a massively complex and tricky operation, taking days or weeks to complete.
Fortunately you don't have to..
And Webmin, of course. They come as a package. And wow! What a package!
I remember Webmin from days of yore. Handy. This Virtualmin/Webmin thing is a whole other level, literally. You to run as many web sites as your hardware can handle. Think hosting panel on steroids. It basically enables complete control of your box, from the intricacies of domain management, right down to the basic system hardware and software setup. And all from a nice blue, mobile-friendly web interface (yes, there are themes).
Once you have your chosen OS installed (your VPS company will have a control panel where you can select and install an OS, or their tech guys will do it for you), you can ssh into your box and install Virtualmin with one single command, well, two, if you include the download..
If you aren't logged in as root, prepend the install command with
I'm not going to go into details about setting up a VPS, because that has already been done excellently elsewhere.* Most big VPS hosting companies have tutorials, too. Nor am I going to run through Virtualmin setup, for the same reason.
Suffice to say, you run the command, Virtualmin does a heap of installs, your console scrolls and at the end of it all, you have a feature-rich, open source hosting control panel that wipes the floor with commercial offerings.
The first time you open it, your mind will be blown, but after a while you get familiar with its many many tabs and features and as you realise the huge amount of work and thought that went into it, it starts to make sense. Incredible stuff. And all free.
Okay, it's not for absolute beginners, but if you are comfortable enough in a console (and with Virtualmin, you rarely, if ever need to use an actual console** if you don't want to) and don't mind doing a bit of research and fiddling, it's certainly within your capabilities. Whatever happens, you will learn loads.
So, to summarise... VPS + Virtualmin = Hosting Nirvana. Doo eeeet.
>_icon. THOUGH NOTE: This is NOT an "interactive" shell, so you can't do stuff that requires ncurses, etc., e.g.
htop. It's basically the same as "Command Shell" module. Useful, but you would be better in a real console.
Google will provide most of the answers you seek. The Virtualmin "community" is pretty large. Here are some things you might not find easily, or at all, but which might prove useful..
One of the first things you need to setup, is backups. Ideally daily, and off-site.
Virtualmin offers multiple backup "modules". Each offers a variety of destinations, one being SSH; so if you have another fast server kicking around, or even a Linux rig in your home network, you can easily get scheduled automatic off-site backups. The main Virtualmin backup modules are..
Webmin > Backup Configuration Files
This makes a backup of all your configuration files and the SSH facility works well. You can do
strftime substitutions in file names and a destination might look something like this..
This being the path on the remote server. In this module there is no facility to delete old backups. That is up to you (they aren't even a MB, so no biggie).
The destination file would look something like:
So far so good. However..
Webmin > System > Filesystem Backup
This makes good old file system backups using
dump, and the SSH does not work. This backup appears to require a quite specific set of unlikely circumstances; for starters, your remote host needs to have
rmt installed. Or some fiddling. Hmm.
At any rate, I gave up on that insecure-looking malarkey fairly quickly and instead used a simple post-backup
scp command to achieve the same effect. This one is for my
/etc filesystem backup..
And that's it. Fully automated filesystem backups to an off-site server.
Hold On! I hear you say. What about the password? Can't be having scripts running in the background asking for passwords!
The answer, of course, is keys*. Login as root to your VPS box, make a key..
ssh-keygen-t rsa -C "USER"
Accept the defaults ("USER" is just a comment, but will help you differentiate the key later - use something that will make the key easy to differentiate in a file of many keys, like "
your-name@your-PC-name"), no password (which would defeat the purpose), then copy and paste the new key out of
~/.ssh/id_rsa.pub and into the end of this file..
That's a file named "
authorized_keys" (which you may need to create) inside a directory named "
.ssh" (which again, you may need to first create) inside your home folder on the remote server, wherever that may be (whichever user you will login as), probably not
So now there's a copy of your VPS-server-published root user key in the
authorized_keys file on the remote host.
Ensure the remote machine's permissions are set to
~./ssh folder and
0600 for the
~/.ssh/authorized_keys file (in case "
StrictModes" is set in the
sshd_config). That's it.
Now the two servers are best friends and your scripts and
cron jobs (and you) can happily
scp and ssh to the remote server all day long without passwords.
More fun with keys..
I should add that once you have keys installed, it's much easier to (do so many things including) setup the above Filesystem Backups using the built-in SSH facility. The only thing that will likely get in your way is the port allocation.
There is no way, from within Webmin, to change the port used by the built-in tar-over-ssh. And who uses port 22 for SSH these days? The devs on the Webmin forum say it cannot be switched from the default. But it can.
A simple work-around (thanks to dev Jamie Cameron at the Webmin bug tracker)..
/root/.ssh/config file containing :
Host backup-destination.IP.address Port 7777
Boom! It works instantly. You would think they would make a button.
NOTE: You can also put host names in this file, so if you had a second VPS, as well as a regular entry, you could fake a nice short entry in
127.0.0.1 localhost localhost.localdomain ::1 localhost localhost.localdomain REAL-IP-OF-OTHER-VPS vps2
Then add that fake host to your
Host vps2 Port 7777
And then when you need to ssh into your other box you simply do..
With no need to specify port number or long domain names or hard-to-remember IP addresses. Do the same thing on both boxes.
Sure, you could make an alias in your
~/.bash_aliases files and reduce any length of command to a few simple keystrokes, but this method works everywhere; in scripts, in your shell, wherever
scp ar used.
Virtualmin > Backup and Restore
The backup third option is an entire section enabling backups to Amazon S3, Cloud storage and more. Here you can backup entire virtual servers and their configuration, or restore the same. The Scheduled Backups sub-section is the one most people will probably want, at least initially.
These backups are in a format that is designed for Virtualmin to easily restore, but almost completely useless for humans as the backup is in a zillion numerically-named folders. You will want to setup a schedule for this, of course; in case of need; but for usable backups that you can pluck files from later, the first two backup options (above) are definitely the way to go. So set this, test it, and forget it.
Note: if you are using separate files for each domain (seems smart) what you enter into the "file" input will be used as the backup directory, so no extension required here, simply do..
And your remote backup will be..
With this backup module, put your ssh port directly after the IP address in the usual format:
Keys are superior to passwords and switching over leap-frogs a half-dozen half-arsed "accepted" security measures in one stroke, and makes communication between your PC and VPS effortless. Just do it. It literally takes less than a minute. Then disable password access altogether and say goodbye to so-called "hacking" attempts.
If you are on Windows, see here. Once you have created your
ssh_private_key.ppk file, you can simply point any program that wants to access your VPS (PuTTY, Kitty, WinSCP, Filezilla, etc.) to that file and never have to think about passwords again.
A quick and obvious note about remote backup security..
It perhaps doesn't go without saying that if you are using a second server for backups, you should NOT be using the root account for these backup operations.
Create a separate user just for backups with permission to only do backups, and create backup directories on the target machine owned by that user. The same username on both VPS seems sensible.
Migrating from Cpanel
This is surprisingly painless. So long as you don't mind having your sites in the same user sub-directory structure as before (and you can change this afterwards).
Virtualmin > Add Servers > Migrate Virtual Server
Then you simply pick your FULL backup file and click. Done.
The downside is how long it might take to upload your full CPanel backup to your new VPS in the first place. We are probably talking many GB. Even if you have super-fast fibre, why tie up your own connexion with this chore? Once again,
scp comes to the rescue..
This example copies a backup directly from the old web server (namecheap, where I have been living this last year) to the root of my shiny new VPS. And it happens in seconds instead of far, far longer. So long as you have SSH access to your old server, this is the way.
Note: you will be prompted for the password to your old server. Don't make keys for that server; you won't be back!
Note: I specified the port with "
-P 21098", because that's what namecheap uses. If your old server uses the default port (22), you don't need to specify the port.
Note: For obvious reasons, you want to do this before your old hosting plan runs out. If you are migrating from namecheap, this is a surprisingly long time after your plan elapses! (10 days)
Free SSL, for mail, too..
Virtualmin makes it a doddle to get a free SSL certificate from Let's Encrypt. This is another thing hosting companies would prefer you paid for.
What isn't immediately apparent is how easy it is to have this certificate copied over to other servers, like postfix; your mail server. There is, in fact, a button for it!
Virtualmin > SSL Certificates [tab] Service Certificates.
So now you can send and receive mail with a valid SSL certificate. At least for one domain.
Currently there is no working SNI in Virtualmin postfix; it's on it's way mid 2021. In the meantime, when you copy over your certificate, remember THERE CAN BE ONLY ONE! Like Highlander. Whichever certificate you copy over (last!) covers all your domains.
It might be best to use your main domain as a single MX for all your domains until SLI arrives.
Where is my Inbox?
This was a weird one. Logging into Usermin, checking mail, I have no inbox. I migrated from CPanel and have been using Thunderbird; everything works fine, but just wanted to check the webmail was working, as I sometimes need it. Hmm.
Turns out the email was set to use a file rather than a folder (IMAP style). I changed it to folder, restart usermin and bingo! All my emails appear.
Being a few levels deep in the prefs, I of course made a bookmark..
Webmin >> Usermin Configuration >> Usermin Module Configuration >> Read Mail :: Mail storage format for Inbox
Fairly soon after I got my VPS up and running I realised that what I really needed, was a second VPS.
For around a tenner a year (+VAT!), you can get..
- Automated off-site, crazy-fast backups (30GB+ space). This is worth the money alone. But you also get..
- A fixed IP that you can permanently allow through an otherwise tight, responsive firewall. If your own (possibly dynamic) IP changes, and for any reason you can't ssh into your server, there is always another path in, with guaranteed access.
- A place to test apps, sites, (nice)bots and servers safely away from your main VPS.
- A place to host games for your kids.. "Daddy, can we run Minecraft on your server?". The answer is now "Yes!" (so long as you have at least 1.5GB RAM on-board!).
- A place to test the impact of upcoming server OS upgrades.
- A place to study and learn alternate operating systems (cuz you want Slackware on there, right!).
- I could go on. And don't forget this is in addition to all the marvellous benefits of having a single VPS. But times two.
A net-connected root shell box is always fun. Two is more-than-double-funner. Try it.
That "tight, responsive firewall"..
Being Linux underneath, Virtualmin has an excellent firewall. iptables. And the web is stuffed with interesting scripts and tweaks to give you useful firewall features. If you have the time and inclination, that is.
However, if you are a busy type, running a busy internet-facing server, I recommend something that can, in a couple of commands, turn your Linux firewall into the firewall-of-death-to-bad-bots..
It takes your Linux firewall to the next level. You want to know when processes are running away, or ports are being scanned or login attempts fail and much, much more, this is the firewall appplication you need. The simple instructions even include a section for webmin (basically, install the module and go).
csf can be initially overwhelming, especially the zillion warning emails. But if you take a few minutes with the config file, you will be able to get these and much more under your control. By the way, you can exlclude processes and commands (php cgi, etc) inside
/etc/csf/csf.pignore. You can even have it automatically allow dynamic domains. Handy!
csf takes a minute to get under control, but once you do, you won't regret it.
Miscellaneous Virtualmin tips, tricks and notes for n00bs..
mc: aka. GNU Midnight Commander
If you are new to the console and Linux in general, and even if you aren't, mc is a superbly useful tool for navigating the filesystem and doing stuff. If it isn't installed, you can get it like this (as
root, or use
#Debian/Ubuntu apt-get install mc #Centos/Redhat yum install mc
Launch mc (by typing
mc<Enter> into your console). Hit F9 (Command menu) > Down arrow > Panel Options > Lynx-like motion (enable). OK. Now you can whiz about the filesystem with the arrow keys, like a ninja.
You will probably also want to enable automatic pull-down of menus (when you hit F9). Hit F9 (Command menu) > Down Arrow (for the last time!)> Configuration > Drop down menus (enabled) so you don't have to hit the down arrow to activate the menus in future. Now you are all set.
- F10 closes anything.
- mc has a built-in editor,
mcedit. Hit F4 when a file is selected to invoke. mcedit has skins, syntax highlighting and HotKeys like mc itself, and is a handy alternative to the likes of nano, vi and Emacs.
- TAB to switch panels. Ctrl+U to swap panels. Ctrl+o to switch to a shell. Carefully explore the menus for lots of other useful HotKeys.
- Ctrl+\ brings up the bookmarks. This gets used a lot.
- The "meta" key menu entries, like "Find Files M-?" you get by doing Alt+? (or Alt+Win+? if you have kde-mover-sizer installed in Windows). The find, by the way, is essentially grep, but without the command-line. Very useful. Be careful searching from root without excluding folders..
- Shortcuts in the menu like "
Symlink C-x s" mean, Hit Ctrl+x, let go, then hit "s".
- Alt+Enter inputs the currently selected item into your current command-line. Very handy. If you have Alt+Enter set to switch you to full screen (KiTTY/PuTTY/etc.) you won't be able to do this. Disable that shit!
- F2 while a file is selected to "do something" to the file. That menu, like everything in mc, is completely configurable. It's also smart enough to pick up any compression tools you might install.
- You can use "Shell link" to open a secure SSH connexion to your second VPS (or another Linux box) using in one of the panels. If you have keys installed, simply do
user@vps2:portin the input and Voila! Now make a bookmark!
- You can use your mouse in mc, if you really must. Keyboard is quicker, though
That should be enough to get you started.
Being Linux, when you login, you get a bash shell. Being a bash shell, it will have a
.bashrc file in your home (root) directory, lovingly known as
~/.bashrc . If you are smart, you will edit this file and the associated
.bash_aliases, if it exists - yes on Ubuntu, no on CentOS - and pimp your VPS login up to the max, or at least make it useful. A few minutes here could save you hours in the future.
By the way, on CentOS and other Linux that don't use the
.bash_aliases convention, you might want to add that, as it's a nice idea to keep your aliases away from your other login commands. Simply add this to your
# Aliases.. if [ -f ~/.bash_aliases ]; then . ~/.bash_aliases fi
Aliases are shortcuts that you create so instead of typing this:
You can simply do..
Or whatever. You want to create aliases for any commands that you need to perform often in a shell; restarting servers, tailing log files, whatever you need to do, except more quickly.
For example, this creates an alias named "
lports" which displays the currently listening (IPV4) ports..
# current listening ports.. alias lports='lsof -Pni4 | grep LISTEN'
So I can do
lports at any time in the console to see the current listening ports. I also have an
lports6, which I guess you could string together with a "
Over the years, one builds up a collection of login script snippets, from the useful to the downright silly. But once you get a set that works, you will want to replicate those to all your VPS. Yes, they will likely need a wee tweak for different platforms, but not a lot** .
A couple of examples..
If you have csf installed, you could add this..
# Restart ConfigServer Security & Firewall (csf).. alias ssec='systemctl status csf;systemctl status lfd' alias rsec='csf -ra;systemctl restart lfd;ssec'
Notice how I used the first alias in the second command.
syslog to have a look at the most recent 200 log lines..
alias syslog='journalctl -b -n 200'
fnd to quickly find with a specified name anywhere in the filesystem..
# QuickFind (TM) alias fnd="find / -path '/proc' -prune -type f -o -name"
And so on.
CAUTION! Before you add an alias, remember to check their isn't already a system command with that name! Many of the cute two-letter combinations will have already been taken. BE CAREFUL!
csfinstalled, there is a useful list of current system binary/log locations at the foot of
Secure remote backups..
The built-in backup functions are fine for local backups or remote backups to servers you control. But if you want to backup to a server where there is a chance others might get to your backups (like a shared server), you will want to at least use a password protected archive. This is fairly simple to achieve..
7z, if you haven't already..
#Debian/Ubuntu apt-get install p7zip-full p7zip-rar #Centos/Redhat yum install p7zip p7zip-plugins
Then make a script, perhaps
#!/bin/bash # prefs pass="MyBackupPassword" #end prefs # Grab it once NOW and re-use.. filex="$(date +%Y_%m_%d@%I_%M_%p)" # enter directory to backup.. (I prefer doing it this way) cd /etc # create a 7zip archive, max strength, using our password from above. 7z a -mx=9 -mhe -t7z -r /backup/etc_$filex.7z * -p$pass # transfer to insecure host.. scp /backup/etc_$filex.7z ME@SomeServer.Com:/home/ME/backup/settings/etc_$filex.7z && rm -rf /backup/etc_$filex.7z # repeat for other directories.. cd /root 7z a -mx=9 -mhe -t7z -r /backup/root_$filex.7z * -p$pass scp /backup/root_$filex.7z ME@SomeServer.Com:/home/ME/backup/NY_VPS/homes/root_$filex.7z && rm -rf /backup/root_$filex.7z
For the scp commands to work, you will need to have keys setup. See above. And use your own details, of course.