Offload Data Transfer (ODX) in Windows Server 2012

I've been working on a nice Dell R720 host based Hyper-V cluster this week with a Dell Compellent array providing the storage.

One of the things I was looking forward to with this job was getting hands on with the ODX feature of the Compellent.

ODX (Offload Data Transfer) is a feature found on some of the newer storage arrays that helps with large file operations by (in simplified terms) keeping the transfers within the array rather than passing the file to the source then destination servers then back to the array.

The first thing to do (assuming you know the hardware supports it) would be to check that the OS and it's software components supports ODX.

Now this is a Windows Server 2012 and 2012 R2 only feature so if you're on 2008 R2, time to upgrade.

From a PowerShell prompt, run the following command:
Fltmc instances

Take a note of the volume name of either the drive, or in my case the CSV volume you want to check.  Then run:
Fltmc instances -v <Volume Name>

e.g. Fltmc instances -v C:\ClusterStorage\Volume1

This will give you the filter names that you will need to check.
Run this command, replacing the <Filter Name> with those shown by the previous command.

Get-ItemProperty hklm:\system\currentcontrolset\services\<FilterName> -Name "SupportedFeatures"

So for my two filters of FsDepends and MpFilter I get the following output:

The property that needs checking is "SupportedFeatures".  If this has a value of 3 then ODX is supported and you're good to go.  Anything else and you'll need to look into it further.

Lastly, check if ODX is enabled on your server using this command:
Get-ItemProperty hklm:\system\currentcontrolset\control\filesystem -Name "FilterSupportedFeaturesMode"

If it returns a "FilterSupportedFeaturesMode" other than 0 as shown below then ODX isn't enabled.

Run this to enable ODX:
Set-ItemProperty hklm:\system\currentcontrolset\control\filesystem -Name "FilterSupportedFeaturesMode" -Value 0 -Type DWord

Or this to disable ODX if needed:
Set-ItemProperty hklm:\system\currentcontrolset\control\filesystem -Name "FilterSupportedFeaturesMode" -Value 1 -Type DWord

In order to demonstrate to the client that ODX was indeed enabled and more to the point worth having, I modified the script on Hans Vredevoort shows on his blog post discussing ODX testing between 3Par and Compellents here: http://www.hyper-v.nu/archives/hvredevoort/2013/07/notes-from-the-field-using-odx-with-hp-3par-storage-arrays/

I ran the script which loops through creating 10 x 50Gb and 10 x 475Gb fixed disks with ODX enabled and then does the same but with ODX disabled.

This was the timings from the test:

With ODX
12.6 seconds for 10 x 50Gb vhdx files
84.2 seconds for 10 x 475Gb vhdx files
96.8 seconds total for all vhdx files

Without ODX
1015.5 seconds (nearly 17 mins) for 10 x 50Gb vhdx files
8615.8 seconds (just over 2 hours) for 9 x 475Gb vhdx files (N.B. I ran out of disk space for the 10th)
9631 seconds or 2.7 hours total for all vhdx files

 

There is a MASSIVE difference in creation times!

ODX is a feature well worth having in my opinion.  What I really can't wait for is ODX support with SCVMM libraries in the SCVMM 2012 R2 release!!


I've uploaded the ODX Test script to SkyDrive here: http://sdrv.ms/16QhZZE

Windows Server 2012 Server Manager "Red Services"

I've been using Server 2012 heavily now since release and in particular the new Server Manager that's thrown in your face whenever you log into a server I've come to find very useful.

However... One thing that always niggled me is by default it alerts you to EVERY service known to man that is stopped.  This is brilliant for alerting you to services that really should be running but it also catches one service in particular that will 99% of the time always be stopped but is set to a startup type of Automatic, the Shell Hardware Detection Service.

The behaviour of this service was modified in Server 2008 to stop when a user logs off to reduce the attack surface.

This results however in a default sea of red for your console.

 

The fix for this is very simple really.

 

Click the Services link in the offending dashboard box, usually the All Servers one.

Drop the list of Services down and look for Shell Hardware Detection and untick it.

 

 

And there we go, a nice filtered display, hopefully green.  At least now when it goes red you know there is actually something that needs attention!

 

This setting will also be saved so you don't have to do this every time you open Server Manager.

Microsoft's "Blue" or R2 Wave

In 9 days we're due to get the Beta versions of Microsoft's next wave of releases (Formerly known as Blue) for Windows 8.1, Windows Server 2012 R2 and System Center 2012 R2.

I'm going to leave Windows (Client and Server) to one side for now as there's enough coverage of those features, but what's publically known in terms of new features and changes for System Center?

So far for System Center 2012, it's been mainly SCVMM features announced, along with some ConfigMgr and Intune details.

Virtual Machine Manager (SCVMM)

• Management of physical network switches via OMI
• OOB Service Templates to provision other System Center 2012 R2 components
• OOB Service Template for Network Virtualisation Gateway with Windows Server 2012 R2
• Use ODX deployment capability from Libraries sharing the same SAN
• First node in a tier can run different scripts - helps deploying guest clusters
• Better Windows Server 2012 IPAM integration
• Shared VHDX support for guest clustering
• Dynamic VHDX resizing
• Linux Dynamic memory support

Configuration Manager (ConfigMgr)

• Deploy and manage Windows 8.1 and Windows Server 2012 R2
• Provision certificates, Wi-Fi and VPN profiles
• Deploy links to web applications
• RBAC Reporting control
• Create and modify offline VHD Images
• Publish VHD to SCVMM for use with templates

Intune

• Auto VPN configuration
• VPN and Wi-Fi profile configuration
• Single pane of glass for both Mobile and Devices via Intune into ConfigMgr
• More MDM policy configuration options
• Selective wipe rather than just full device wipe
• The new Server 2012 R2 Work Folders feature configuration

I suppose we're just going to have to wait until next week to get our grubby mits on more information and the ability to have a play.

Seize FSMO roles in Server 2012

One of the beautiful things of a test lab is getting to try things you might not get chance to do in a production environment.  So when my main Domain Controller went pop the other day, rather than work on bringing it back online I saw a good chance to test seizing the FSMO roles with PowerShell.

Previously the main way to seize the roles was using the Ntdsutil in Server 2003 & 2008.

Since PowerShell is now my weapon of choice I thought it would be useful to quickly document the method.

Move-ADDirectoryServerOperationMasterRole is the command that is used for this task.  More information on the command can be found here:
http://technet.microsoft.com/en-us/library/ee617229.aspx

You can use either the Role Name or Number to specify which role to move, this table shows the details:

Operation Master Role Name	Number
PDCEmulator	0
RIDMaster	1
InfrastructureMaster	2
SchemaMaster	3
DomainNamingMaster	4

 

 

Use the -Identity switch to specify the target Domain Controller and the –OperationMasterRole to specify which role to transfer. I've also used the -Force command as my current FSMO holder is offline.

 

I'll be moving all the roles to a target DC called TLDC02.
N.B. To move the SchemaMaster role you'll need to be a member of the Schema Admins group.  My account was also a member of Enterprise Admins when I ran this.

1. Logon to a working Domain Controller and launch an elevated PowerShell session.
2. Type: Move-ADDirectoryServerOperationMasterRole -Identity TLDC02 -OperationMasterRole 0,1,2,3,4 -Force





3. Either type Y on each role move prompt, or type A to accept all prompts
4. After a while, all the roles should be successfully moved.

Last thing, a couple of PowerShell command just to list the FSMO roles and who now owns them:

Get-ADForest DomainName | FT SchemaMaster,DomainNamingMaster
Get-ADDomain DomainName | FT PDCEmulator,RIDMaster,InfrastructureMaster

One thing to note, only seize the roles if you have no intention of bringing the original holding Domain Controller back online.  Domains don't tend to like having two FSMO role holders...

Problems Clustering Virtual Machines on Windows Server 2012 Hyper-V

I was re-building our lab environment at work the other week in preparation for our big Summit13 event, that and the lab had been trashed over the last year...

As part of the re-build I had decided to implement a couple of virtual machine clusters, one for a scale-out file server and one as a SQL cluster.

I'd deployed the virtual machines for the cluster nodes using Service Templates in SCVMM and as part of that template chosen to use an availability set to ensure the VM's were separated across hosts (a cluster doesn't provide much High Availability if they all reside on the same host that's failed!).

When I started to create the cluster I ran straight into a problem with the Failover Cluster Manager reporting problems due to timeouts when creating the cluster.

Creating a single node cluster worked fine, but then would again fail when trying to add in another node.

I happened to put one of the Hyper-V hosts into maintenance mode for something and it migrated all the VM's onto the same host, at which point creating the cluster worked flawlessly, yay!

However, when the Hyper-V host came back out of maintenance mode and the availability sets kicked in during optimisation forcing a VM node back onto a separate physical host, the clusters broke again, not yay :(

So after some Googling Binging about and a shout on Twitter (Thanks @hvredevoort and @WorkingHardInIT) an issue with Broadcom NICs was brought to my attention and I came across this TechNet Forum post talking about the same issue.

Sophia_whx suggested trying to use Disable-NetAdapterChecksumOffload on the NICs to help with the issue.

So, first things first.  Use the Get-NetAdapterChecksumOffload to see just what the configuration was and sure enough Checksum Offload is enabled for just about all services across the majority of the NICs.

I then used the Disable-NetAdapterChecksumOffload * -TcpIPv4 command which resulted in this:

 

A reboot later and then perform it on the second host and whoa....

For some reason, the virtual switch really didn't like having that done to it.

I wish I had some screenshots, but I went into "get it fixed fast" mode.

 

Basically the switch via powershell was showing as up, the NIC Teaming GUI was showing it down and all the bound adapters as failed. SCVMM had lost all configuration for the switch altogether.

Deleting the switch from SCVMM didn't delete it from the host, but brought it back to life on the host but was missing in SCVMM.  SCVMM then wouldn't redetect it or let you build it again as it was still there, apparently???

 

I had to manually remove the team from a remote NIC Teaming GUI (I could of PowerShell'd it I know!) and then recreated via SCVMM.
 

Anyway... at first this looked to have fixed the clustering within virtual machine issues, but it only delayed the symptoms i.e. it took longer to evict nodes and randomly brought them back online.

 

So next I tried disabling Checksum Offload for all services, being careful not to touch the Virtual Switch this time.

 

Rather than going adapter by adapter I used the following command:

 

Get-NetAdapter | Where-Object {$_.Name -notlike "Converged*"} | Disable-NetAdapterChecksumOffload

 

This resulted in the Checksum Offload being disabled for the various services as shown, except for my virtual switch.

After doing this on the other host and giving them a reboot, my clustered virtual machines appear to be nice and stable when split across physical hosts. Yay! Problem fixed.

Just as another side note about Broadcom adapters, there have also been reports of performance issues when the Virtual Machine Queue (VMQ) setting is enabled, despite it being a recommended setting.

A quick check of my hosts showed it was enabled:

Another quick PowerShell line later and it wasn't:

Get-NetAdapterVmq -InterfaceDescription Broad* | Disable-NetAdapterVmq

Windows Server 2012 - Licensing and Key Features Webinar

I thought I'd throw this one out there...

For anyone involved in Server or Datacenter administration, design, support or management my employers Trustmarque will be hosting an online webinar to discuss and show the new licensing changes that come with Windows Server 2012 along with an overview of some of the new key features of which I'll be co-presenting.

Signup for the webinar here:

http://www.trustmarquesolutions.com/news/overview-of-microsoft-licensing-changes-for-2012/

Windows Server 2012 was Released To Manufacturing (RTM) yesterday (01/08/2012) and Microsoft have announced September 4th as the general release date for Windows Server 2012, bringing with it a much simpler model and many enhanced features. To help you learn more about these changes Trustmarque are running a webinar on Tuesday 7th August at 10.30am and 3pm giving you both a licensing and technical insight.

• Overview of Licensing Changes for 2012 and impact on Software Assurance – Dominic McHugh, Software Procurement Consultancy, Trustmarque
• Features and toolsets improvements and changes – Steve Beaumont and Wayne Robinson, Technical Architects, Microsoft Integrated Solutions, Trustmarque

The webinar will highlight some of the following new or improved features:

• Deployment & automation
• Active Directory Improvements
• Dynamics Access Control – Claims based authentication
• Server Virtualisation – Hyper-V 3.0
• Virtual / Remote Desktop delivery
• Cluster enhancements
• File Server enhancements
• Data De-duplication

Windows 8 & Server 2012 RTM

Well Microsoft said the first week of August for the RTM version of Windows 8 and Server 2012 and it's arrived!

Well kind of...

The Release To Manufacturing (RTM) was announced today and see's the final code wrapped up, finalised and is now starting to get shipped to Microsoft hardware partners (HP, Dell, Lenovo etc) who will finish their final testing ready to deliver devices with Windows pre-installed as from October 26th (September for servers with Server 2012).

What this means for us mere mortals and IT-Pros is that we still have to wait some more.

From the Windows Team blog the schedule looks like this:

• August 15th: Developers will be able to download the final version of Windows 8 via your MSDN subscriptions.
• August 15th: IT professionals testing Windows 8 in organizations will be able to access the final version of Windows 8 through your TechNet subscriptions.
• August 16th: Customers with existing Microsoft Software Assurance for Windows will be able to download Windows 8 Enterprise edition through the Volume License Service Center (VLSC), allowing you to test, pilot and begin adopting Windows 8 Enterprise within your organization.
• August 16th: Microsoft Partner Network members will have access to Windows 8.
• August 20th: Microsoft Action Pack Providers (MAPS) receive access to Windows 8.
• September 1st: Volume License customers without Software Assurance will be able to purchase Windows 8 through Microsoft Volume License Resellers.

I was all prepped to rebuild our Customer Experience Center this week or next with RTM but now it looks like it will have to wait until I come back from holiday, which happens to be the week when the downloads become available (How inconsiderate Microsoft!)

Windows Server 2012 - Quick Change Overview List

There is a wealth of information coming out at the moment regarding Windows Server 2012 and rather than doing an indepth post of all the changes, I thought I'd just call out some of the most interesting changes and new features that have caught my attention.
I've also linked some useful sites that contain some more indepth discussions.

General Windows Changes

• Native NIC Teaming
• SMB 3.0
• Ability to switch between Server Core and Full UI
• Multi Server Management from Server Manager
• Metro interface (present, but not in your face at boot)
• Storage Spaces
• Everything doable in the UI can be done with PowerShell 

Infrastructure Changes

• Domain Controller Replication (Clone & Deploy a DC)
• Virtual Domain Controller Support
• Storage across Remote SMB 3.0 File Shares (SQL & VM's run from file shares)
• IP Address Management
• Dynamic Access Control (File & Folder permissions on steriods!)
• RemoteFX without a hardware GPU
• RemoteFX in RDS Sessions, not just VDI

Hyper-V Changes

• Shared Nothing Live Migration
• Live Storage Migration
• Multiple Concurrent Live Migrations
• Hyper-V Replica
• Extensible Virtual Switch
• SR-IOV
• Network Virtualisation (servers running on the same IP without conflict)

Hyper-V Scalability Changes

• 160 Logical Processors per Host
• 2Tb RAM per Host
• 32 vCPUs per VM
• 512Gb per VM
• 16Tb VHDx
• 63 Nodes per Cluster
• 4,000 VMs per Cluster

Hyper-V 3.0 versus vSphere 5
While comparing an unreleased product to a released product isn't exactly fair, I thought I'd include it for completeness sake.  Also it's the most common question that customers ask me.
This article has a fairly clear comparison table:
http://up2v.nl/2012/01/29/microsoft-hyper-v-3-0-compared-to-vmware-vsphere-5/


Information Links
http://www.aidanfinn.com/ - Massive resource for Hyper-V
http://workinghardinit.wordpress.com/ - Another huge Hyper-V resource

http://blogs.technet.com/b/server-cloud/archive/2011/10/11/windows-server-8-hyper-v-overview.aspx

http://www.microsoft.com/en-us/server-cloud/