Scripted CIFS Shares Migration

I don’t usually blog about Windows Server and Microsoft products in general, but the need for file server migration comes up in my work quite frequently, so I thought I’d make a quick post on that topic.

There are many use cases, it can be migration from a NAS storage array to a Windows Server or between an on-premises file server and cloud. Every such migration involves copying data and recreating shares. Doing it manually is almost impossible, unless you have only a handful of shares. If you want to replicate all NTFS and share-level permissions consistently from source to destination, scripting is almost the only way to go.

Copying data

I’m sure there are plenty of tools that can perform this task accurately and efficiently. But if you don’t have any special requirements, such as data at transit encryption, Robocopy is probably the simplest tool to use. It comes with every Windows Server installation and starting from Windows Server 2008 supports multithreading.

Below are the command line options I use:

robocopy \\file_server\source_folder D:\destination_folder /E /ZB /DCOPY:T /COPYALL /R:1 /W:1 /V /TEE /MT:128 /XD “System Volume Information” /LOG:D:\robocopy.log

Most of them are common, but there are a few worth pointing out:

• /MT – use multithreading, 8 threads per Robocopy process by default. If you’re dealing with lots of small files, this can significantly improve performance.
• /R:1 and /W:1 – Robocopy doesn’t copy locked files to avoid data inconsistencies. Default behaviour is to keep retrying until the file is unlocked. It’s important for the final data synchronisation, but for data seeding I recommend one retry and one second wait to avoid unnecessary delays.
• /COPYALL and /DCOPY:T will copy all file and directory attributes, permissions, as well as timestamps.
• /XD “System Volume Information” is useful if you’re copying an entire volume. If you don’t exclude the System Volume Information folder, you may end up copying deduplication and DFSR data, which in addition to wasting disk space, will break these features on the destination server.

Robocopy is typically scheduled to run at certain times of the day, preferably after hours. You can put it in a batch script and schedule using Windows Scheduler. Just keep in mind that if you specify the job to stop after running for a certain amount of hours, Windows Scheduler will stop only the batch script, but the Robocopy process will keep running. As a workaround, you can schedule another job with the following command to kill all Robocopy processes at a certain time of the day, say 6am in the morning:

taskkill /f /im robocopy.exe

Duplicating shares

For copying CIFS shares I’ve been using “sharedup” utility from EMC’s “CIFS Tools” collection. To get the tool, register a free account on https://support.emc.com. You can do that even if you’re not an EMC customer and don’t own an EMC storage array. From there you will be able to search for and download CIFT Tools.

If your source and destination file servers are completely identical, you can use sharedup to duplicate CIFS shares in one command. But it’s rarely the case. Often you want to exclude some of the shares or change paths if your disk drives or directory structure have changed. Sharedup supports input and output file command line options. You can generate a shares list first, which you can edit and then import shares to the destination file server.

To generate the list of shares first run:

sharedup \\source_server \\destination_server ALL /SD /LU /FO:D:\shares.txt /LOG:D:\sharedup.log

Resulting file will have records similar to this:

#
@Drive:E
:Projects ;Projects ;C:\Projects;
#
@Drive:F
:Home;Home;C:\Home;

Delete shares you don’t want to migrate and update target path from C:\ to where your data actually is. Don’t change “@Drive:E” headers, they specify location of the source share, not destination. Also worth noting that you won’t see permissions listed anywhere in this file. This file lists shares and share paths only, permissions are checked and copied at runtime.

Once you’re happy with the list, use the following command to import shares to the destination file server:

sharedup \\source_server \\destination_server ALL /R /SD /LU /FI:D:\shares.txt /LOG:D:\sharedup.log

For server local users and groups, sharedup will check if they exist on destination. So if you run into an error similar to the following, make sure to first create those groups on the destination file server:

10:13:07 : WARNING : The local groups “WinRMRemoteWMIUsers__” and “source_server_WinRMRemoteWMIUsers__” do not exist on the \\destination_server server !
10:13:09 : WARNING : Please use lgdup utility to duplicate the missing local user(s) or group(s) from \\source_server to \\destination_server.
10:13:09 : WARNING : Unable to initialize the Security Descriptor translator

Conclusion

I created this post as a personal howto note, but I’d love to hear if it’s helped anyone else. Or if you have better tool suggestions to accomplish this task, please let me know!

Advertisement

How to move aggregates between NetApp controllers

 

DISCLAMER: I ACCEPT NO RESPONSIBILITY FOR ANY DAMAGE OR CORRUPTION OF DATA THAT MAY OCCUR AS A RESULT OF CARRYING OUT STEPS DESCRIBED BELOW. YOU DO THIS AT YOUR OWN RISK.

 

We had an issue with high CPU usage on one of the NetApp controllers servicing a couple of NFS datastores to VMware ESX cluster. HA pair of FAS2050 had two shelves, both of them owned by the first controller. The obvious solution for us was to reassign disks from one of the shelves to the other controller to balance the load. But how do you do this non-disruptively? Here is the plan.

In our setup we had two controllers (filer1, filer2), two shelves (shelf1, shelf2) both assigned to filer1. And two aggregates, each on its own shelf (aggr0 on shelf0, aggr1 on shelf1). Say, we want to reassign disks from shelf2 to filer2.

First step is to migrate all of the VMs from the shelf2 to shelf1. Because operation is obviously disruptive to the hosts accessing data from the target shelf. Once all VMs are evacuated, offline all volumes and an aggregate, to prevent any data corruption (you can’t take aggregate offline from online state, so change it to restricted first).

If you prefer to reassign disks in two steps, as described in NetApp Professional Services Tech Note #021: Changing Disk Ownership, don’t forget to disable automatic ownership assignment on both controllers, otherwise disks will be assigned back to the same controller again, right after you unown them:

> options disk.auto_assign off

It’s not necessary if you change ownership in one step as shown below.

Next step is to actually reassign the disks. Since they are already part of an aggregate you will need to force the ownership change:

filer1> disk assign 1b.01.00 -o filer2 -f

filer1> disk assign 1b.01.01 -o filer2 -f

…

filer1> disk assign 1b.01.nn -o filer2 -f

If you do not force disk reassignment you will get an error:

Assign request failed for disk 1b.01.0. Reason:Disk is part of a failed or offline aggregate or volume. Changing its owner may prevent aggregate or volume from coming back online. Ownership may be changed only by using the appropriate force option.

When all disks are moved across to filer2, new aggregate will show up in the list of aggregates on filer2 and you’ll be able to bring it online. If you can’t see the aggregate, force filer to rescan the drives by running:

filer2> disk show

The old aggregate will still be seen in the list on filer1. You can safely remove it:

filer1> aggr destroy aggr1

Moving DFS Replicated Folders in Windows Server 2003

If you run out of space on DFS replication target LUN you can’t simply provision another LUN and copy data over. Because it will change files attributes. There are two ways to preserve files metadata: use robocopy or NTBackup tool. But using robocopy is not recommended to pre-seed replicated folders in Windows Server 2003, because there are some issues in the version included in Windows Resource Kit Tools.

The way to go here is to use NTBackup. Remove the folder from DFS configuration first, to stop replication. Then backup the replicated folder and restore it to whatever LUN you want it to be on. To check that everything’s done right use dfsrdiag utility with filehash option. Dfsrdiag which is included in Windows Server 2003 doesn’t support filehash option. So you will most likely need to install recent RSAT tools for Windows 7 or Windows 8 on your desktop. Use dfsrdiag on several files and if it returns the same hashes, then it’s safe to assume that all other files were restored correctly too. Here is the example:

dfsrdiag filehash /path:”original path to the file”
dfsrdiag filehash /path:”path to the restored copy”

As the last step, add the new folder path to the DFS replication group. It will take some time for DFS to compare all the data. But if everything has been done right, only the delta will be replicated between members.

Migrating physical Linux host to the VMware ESXi

Well, perhaps the easiest way to accomplish that is using VMware Converter from the start. I believe there is a Linux version. However, I took another route. I already had an Acronis backup image. So my solution was to use this image as a source, which I fed into Windows version of VMware Converter, which in its turn converted it to VMware format and created VMware virtual machine on ESXi server automatically.

Using this simple procedure you can get a working system. Not in my case. Original OS used a software RAID of two hard drives. So I had to boot from a live CD. Then I changed fstab and GRUB’s menu.lst and set /dev/sda1 (root volume) instead of /dev/md0 and /dev/sda2 (swap) in place of /dev/md1. Additionally, I had to reinject GRUB’s boot files:

grub-install –root-directory=/media/sda1/boot /dev/sda

Then, if it’s SUSE you will have to change “resume” switch in GRUB’s boot menu line to /dev/null. Then after you boot into the system, recreate swap partition and point to it in “resume” switch. If you won’t do that, you will end up with the following error during boot process:

Kernel panic – not syncing: I/O error reading memory image

One tricky issue I had in all this story was related to kernel. As I’ve already mentioned original operating system worked on top of software RAID. And its initrd image won’t detect ordinary virtual SCSI hard drive during boot. So I had to boot from the SUSE installation CD and install standard kernel on top of original system. It solved the issue. Additionally I had to choose Russian language as primary during kernel installation, otherwise I ended up with unreadable symbols inside the system. But it’s not necessary for majority of cases.

I hope my experience will be helpful for other sysadmins.

Moving Active Directory roaming profiles to another server

Relocating Active Directory roaming profiles can be a tricky task. You have many shared folders with particular permissions which won’t move unlike NTFS permissons to another server if you simply copy these folders. On top of that, you need to change profile paths in Active Directory Users and Computers snap-in. And if you have hundreds of users it’s not what you will happy to do. Given these two objectives lets move on to implementation.

Moving shares preserving permissions

I ran into several suggestions how to do that, like using robocopy, xcopy, permcopy or other tools. I don’t know to what extent they might help. I’d like to suggest simpler solution. Microsoft has  File Server Migration Toolkit (FSMT). It’s very basic and limited tool. It means you will probably need to do some hand work. But it solves the primary problem which is copying shares along with their permissions.

FSMT has additional feature of creating DFS links for you but I didn’t use it. GUI is rather intuitive, so there is not much to explain. The particular problem with FSMT is that it changes target share and folder paths. Say you have share with the name ~UNAME$ which is located on server CONTOSO_PDC. As a result of movement you will have share with the name ~UNAME$_contoso_pdc$. Which is not what we expect to have in our case. Same thing for target folder. For example, if the source folder for the share is D:\Profiles\UNAME, after migration you’ll get D:\Profiles\contoso_pdc\~UNAME$. Apart from additional folder in between, as you can see last part of source path is changed to share name in the target path (~UNAME$ instead of UNAME).

In my case I had to revert all these changes back to what it originally was. The trick here is to create FSMT project, add server and shares to it and then exit without performing the actual move. Then open project .xml file and correct all paths by search/replace. Since I had complicated share names I had to use replace with substitution feature in MS Office Word. For example to change target path from D:\Shared\~PROF\~UNAME$ to D:\Shared\~PROF\UNAME I used following masks for search and replace:

D:\\Shared\\~USER\\\~(*)\$

D:/Shared/~USER/\1

Here word processor searches for the first string and use word from parenthesis as the substitution for the special sequence \1.

Changing profile paths in Active Directory

Here you also have several ways to accomplish that. You can use ADModify tool. But I simply wrote a Powershell script which I share with you as it is. I believe it’s mostly self-explanatory. For convenience I also uploaded this script to FileDen. Download it from here.

# Bind to the root of the current domain
$ldapPath = "LDAP://ou=Users and Computers,dc=contoso,dc=com"
$objDomain = New-Object System.DirectoryServices.DirectoryEntry($ldapPath)

$objSearcher = New-Object System.DirectoryServices.DirectorySearcher
$objSearcher.SearchRoot = $objDomain
$objsearcher.Filter = ("(objectCategory=User)")
$colResult = $objSearcher.FindAll()

foreach($objResult in $colResult) {
	$user = $objResult.GetDirectoryEntry()
	write-host "For user" $user.cn ":"

	$profilePath = $user.ProfilePath
	$parts = $profilePath.ToString().Split("\")

	# Identifying profile type (XP = 0 or Windows 7 = 1)
	if($parts.Length -eq 4) { $profType = 0 }
	elseif($parts.Length -eq 5) { $profType = 1 }

	# Constructing new profile paths
	if($profilePath) {
		if($profType -eq 0) {
			$newProfPath = "\\SERVERNAME\" + $parts[3];
			$newProfDirPath = "D:\~PROF\" + $user.sAMAccountName
			$newProfShareName = $parts[3]
		}
		# Windows 7 profiles do not have individual shares. There is
		# one share for all roaming profiles.
		elseif($profType -eq 1) {
			$newProfPath = "\\SERVERNAME\Profiles\" + $parts[4]
		}
	}

	# Constructing new home directory paths
	$homeDirectory = $user.homeDirectory
	$parts = $homeDirectory.ToString().Split("\")
	if($homeDirectory) {
		$newHomePath = "\\SERVERNAME\" + $parts[3];
		$newHomeDirPath = "D:\~USER\" + $user.sAMAccountName
		$newHomeShareName = $parts[3]

	}

	if($profilePath) {
		# Changing profile path
		write-host "Changing profile path from" `
			$user.ProfilePath "to" $newProfPath
		$user.ProfilePath = $newProfPath
	}
	if($homeDirectory) {
		# Changing home directory path
		write-host "Changing home directory path from" `
			$user.homeDirectory "to" $newHomePath
		$user.homeDirectory = $newHomePath
	}
	# Commit changes
	$user.setinfo()
}