How-to : Build a “mirror / test server”

pci

Introduction

This post is a quick how-to build a test framework for a production server, using either a virtual machine or a good old physical computer. I wrote this how-to when i had to build a test server for myself.
I do own a home server, on which i host several services and i definitely need an identical server to run updates and new service hosting before implementing them on my “production” server.
The following is one of the many way to achieve this task, please feel free to add your own experience.

 

 

 

1 The context

My “production server” is running under a CentOS 6.3, it is used as a classic web server way using apache and MySQL.

My “test server” is a virtual machine running under Virtual Box 4 (check this post to install it).

 

2 The process

As usual you may backup your production server, even if what we are about to do is not critical, an error is still possible (such as inverting the source and destination during a copy process etc…), moreover a recent backup is nothing harmful ! (even if i am sure you do have an efficient and validated backup policy !)

2.1 Overview

Here are the head lines of the process :

  1. Backup the important files from the production server
  2. Create a virtual machine with the same specification as your production server
  3. Copy data (overwrite) from production server to test server
  4. Fix network related issues (network interface probing, hostname and IP address…)
  5. Enjoy your newly created test server !

 

 

3 Let’s do the job !

3.1 Creating the virtual machine

This could be done using any virtual machine software. I did it using Virtual Box (check this post to install it).

I am not going to go through the creating steps as it is widely described on the web, and as it is actually very user friendly.

3.1.1 Important points

When choosing the partition scheme be sure to set-up some partitions at least as big as the original ones (the ones from your production server), the important partitions are :

  • /var : This is where your databases and web server root directories are stored (when using default location)
  • / : The root directory contains all the operating system files and directories
  • Any other partitions you personally set-up on its own partition

The data partition (e.g : the partition on which are stored movies and / or pictures) are not to be identically created, it is unnecessary and definitely not cost-effective.

3.2 Checking your virtual machine

Once your virtual machine is done, you need to ensure it is working as expected. You may then want to login and setup a working network configuration (we are going to use the network to reach the production server).

if everything is ok, we are now about to break it all !

3.3 Copying the production server setup to the virtual machine

As said before we need both computers (production server and virtual machine) to be reachable, once you achieve this you only have to issue the following command on your production server :

tar --exclude=*fstab --exclude=*hosts --exclude=70-persistent-net.rules --one-file-system -Pczf - / | ssh root@<test_server_IP> 'tar xvzf - -C /'

I am using the tar command because :

  1. It is quicker to transfer data over the network by compressing the flow
  2. It keeps the permissions and owner / group informations as it is
  3. It is easy to exclude what needs to be using an exclude statement
  4. It is also easy to exclude other filesystem (e.g filesystem containing your “big data”)

Another efficient command would be rsync

3.3.1 The command explained

– The results of the tar command is sent to the STDOUT (using the - descriptor) and then i redirect the STDOUT to the ssh command using the pipe ( | ) (you may want to check this post for more tips about the possible ssh + tar combination), finally i untar the flow to the required directory (-C /), which in this case is the root directory.

I choose to exclude some file, here is why :

  • The /etc/fstab file : this file contains informations about your local and remote filesystems, some distributions uses UUID to describe hard drives and partitions and UUID are unique
  • The /etc/hosts file : This file contains informations about the “known” hosts and their respective associations between ip addresses and hostnames, you might not want your test server to have the exact same name as your production server in this case you may add
    --exclude=*hosts

    to the previous command line, otherwise there is no need to exclude this file.

  • The /etc/udev/rules.d/70-persistent-net.rules is another file you may want to exclude in order to avoid any network interfaces issue :
    --exclude=70-persistent-net.rules

By using this command line we eventually overwrite the whole root directory of the virtual machine by the production server one.

 

4 Fixing the residual problems

You may still face some problems, here is a list for some of them, if you faced one that is not listed here please ask for a solution using the comment section and i will add them (problems + fix) to this list.

4.1 My network card does not work

When trying to setup the test server network you may encounter a “the eth0 does not exist …” message (when restarting the network using the service network restart for example), in this case you may try the following :

cd /etc/udev/rules.d 
mv 70-persistent-net.rules /root/ 
reboot

This command will force the system to generate a new rule file for the network interfaces.

 

Conclusion

Using this technique you will have a “full” copy of your prod server, testing on this server might requires some more tuning, such as setting up special virtual network to avoid duplicate server on your network and so on.

You may want to keep a “master” using the “snapshot” tools from your virtual machine software and then replicate it to test new setup and services. Another important thing it to try to keep your test server up-to-date regarding your production server.

I hope this answer some question and will help some people getting secure production process.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site supports SyntaxHighlighter via WP SyntaxHighlighter. It can highlight your code.
How to highlight your code: Paste your code in the comment form, select it and then click the language link button below. This will wrap your code in a <pre> tag and format it when submitted.