Condor Airways needed an upgrade and fast.
IBM Cloud were removing their support for the ESXi 6.7 VMware vSphere cluster and the upgrade to version 7.0 had to be completed before the end of October 2022. An in-place migration was also not supported. IT Naturally manage several key components of Condor’s IT Infrastructure in a VMware vSphere cluster hosted on IBM Cloud, so it was our job to offer our customer a suitable solution in a supported environment.
For those less techy reading this, VMware vSphere is a hyperconverged infrastructure which runs virtual machines and virtual servers such as services running as virtual machines or virtual servers include booking and commercial applications; firewalls; Active Directory; security information and aircraft maintenance systems.
The Business Challenge
IT Naturally had to offer a solution to Condor Airlines to:
- Migrate 100+ servers to a completely new physical infrastructure without any downtime
- No changes in Server IP addresses to ensure there was no downtime and applications/clients would not need reconfiguring
- Ensure networks are available in both the legacy and new VMware environments at the same time to allow a seamless migration
- Ensure backup and recovery processes continue so that recovery options are available in case of any migration issue
- Complete the move before the customers peak business period
New environment and migration design
Our Platform and Network teams worked tirelessly to produce the best solution for our customer. Several technical issues had to be dealt with before those solutions could be put in place:
Lee Gray, Senior Platform Engineer and James Poole, Senior Network Engineer reveal the difficulties they faced:
The new VMware environment was completely separated from the legacy VMware cluster, both being controlled by separate vCentre management servers, and each had their own NSX virtual networking. Also, storage datastores where the virtual machines were running were independent of each other. Both environments also had their own SSO (single sign-on) domains to complicate things further.
Various migration approaches were considered including sharing the server datastores between the two clusters, but that would have resulted in downtime for every virtual server while it was migrated to the new environment which we were trying to avoid if technically possible.
One of the trials we also had to overcome during the migration was the switch from NSX-V to NSX-T for the virtual networking infrastructure.
Due to the differences in how the old and new versions of NSX function from both the underlying connectivity technologies and topology design perspective a lot of research and testing was needed to ensure the new environment was fit for purpose.
The connectivity technology difference between the two NSX versions presented complications when linking the two environments, NSX-V using VXLan for L2 tunnelling and NSX-T using GENEVE meant it wasn’t possible to directly connect NSX-V edges to the new NSX-T edges.
Two approaches were found to overcome this, the first was a more traditional networking choice of assigning a VLAN to each NSX-V vSwitch and NSX-T Segment pair then spanning all the new VLANs to each of the old and new physical hosts, the alternative choice was to build new NSX-T Edge VM’s in the both the old and new environments to provide GENEVE “bridges” between the old vSwitchs and new network segments.
Initially we were learning towards the VLAN approach, however as our knowledge and experience grew with NSX-T we instead implemented new edge VM’s to bridge the environments. This had some benefits from a cost perspective as each VLAN within the IBM cloud has a monthly charge associated with it, but it also simplified the post migration clean-up work for us.
Due to all these technical challenges a Cross vCentre vMotion migration approach was taken.
This involved adding an extra Cross vCentre vMotion utility virtual server to the legacy VMware environment. All server migrations were then controlled from this server which communicated with both vCentre servers to start the migration copy of server data disk files to the new datastores then seamlessly bring the migrated server live on the new cluster, all happening when the servers and services were online and running as normal with no downtime or even interruption.”
It was a very successful project, lots of work on our side but with the customer not noting a thing on their side.
Not only was their no interruption at all or downtime, this particular migration approach allowed for physical server hosting costs to be kept to a minimum during the migration too. By starting small with two servers initially in the cluster with half their target memory, then these were increased throughout the migration when the capacity was needed. Storage costs were also reduced by provisioning smaller consolidated datastores in the new environment.
Communication
As with every project or change at IT Naturally, we followed our standard change management process and utilised our 24×7 Service Desk to manage and track the migrations. This made sure progress remained on track and escalations were made if issues came up. IT Natural’s Service Manager liaised with the customers application owners to provide post migration testing.
Customer Outcomes
This project delivered a transition to a new VMware hosting platform with three new physical servers running VMware ESXi 7.0 and NSX-T virtualised networking.
IT Naturally successfully:
- Managed to keep a seamless continuation of services allowing the critical infrastructure to continue to run in the IBM Cloud.
- A cost reduction in Microsoft licensing as the number of CPU cores would be reduced.
- Cost reductions from consolidating storage
- Running a fully supported environment
Conclusion
Following the detailed planning and testing phases, the migration was a huge success moving the customers entire virtual infrastructure (approximately 100 servers) to a completely new hardware environment within 14 days. It was so seamless; the business was not even aware the migration had completed.
