Transcription
How to OptimizeYour Migration SpeedWhen Using SharegateAn eBook by Sharegate
IntroductionIt’s finally time to get your SharePoint migrationSharegate is a powerful and easy to use tool, but it canunderway. You’ve followed all of the instructions in ouronly go as fast as the infrastructure it has to interact with.Migration Checklist to make sure you’ve rememberedeverything and you’re ready to rock. Management hasBy the end of this eBook, you will know how Sharegateset a strict time table for your project and you want tointeracts with different components and what bottlenecksmake sure that you squeeze every possible ounce ofto look out for that could negatively impact the speedperformance out of Sharegate to get everything movedof your migration project.as quickly as possible.About the AuthorAs Chief Technology Officer and one of the mastermindsbehind Sharegate, Christian Merat leads the long termtechnology vision. He’s been working with Sharegatesince day one and is also a tutor for junior developers.2
Overview of the ComponentsInvolved in Your MigrationSharegate sits smack in the middle of your source and destination farms.This means that issues that occur at either of these locations (or even betweenthese locations) can negatively impact the migration speed.Let's have a look at theses different components and what you needto keep an eye out for.3
Source and Destination FarmsIt’s essential to have a healthy farm in orderto obtain optimal performance out of your migration.The entire migration process with Sharegate goesthrough the public APIs that are exposed by SharePointover HTTP/S. All of the interactions go through theWeb Front-End (WFE), some Application Services andfinally the SQL database that holds all of SharePoint'scontent. In the case where Remote Blob Storage (RBS)is used in your farm, this service will also be solicited.To make sure that your farm is in top shape, hereare a few things you can monitor: The SharePoint Health Analyzer isn't flaggingcritical issues.the best practices outline in the SharePointThe servers in your farm do not havefarm is running in tip-top shape.any physical bottlenecks (see section onResource Monitor).4Tip: Before starting your migration, followPerformance Guide to make sure that your
Active DirectorySharePoint has a fairly strong coupling with ActiveDirectory and during a migration, this link is stressedeven more. This is because Sharegate will be doinga lot of user searches and resolutions to properlyassign all of your permissions and author informationto the right users.This will usually result in a large number of callsto Active Directory. So, if your AD server is slowor unresponsive, this can cause a significant decreasein migration speed.Sharegate does have its own internal cachingmechanism, but it will still need to perform eachlookup at least once.5
SharegateThe workstation on which Sharegate is running3- Have the right number of CPU cores on youris going to be doing a lot of heavy lifting, since allworkstation (for a VM, make sure you have enoughof the data that transits from your source farm to yourvirtual processors).destination farm will pass through Sharegate. The number of processors also impacts the numberThere are steps to make sure that this workstationof concurrent items that will be migrated dependingwon't be the bottleneck in your migration:on your Performance Level.1- Depending on your environment, select the right Sharegate is optimized to run with 4 cores(64 concurrent threads) so you should aim for that.Performance Level in the Sharegate options.While it might be tempting to go "High Performance",4- Don’t start many parallel migrations on the samethis will put a lot more strain on your source farm andworkstation (this includes PowerShell).can, in some cases, even reduce overall performancethrough excessive context switching. Try a few different These will fight for resources on the same workstationperformance levels out to find the one that works bestand can potentially lead to significantly decreasedfor your environment.performance. We suggest to use multipleworkstations running Sharegate in parallel.2- Check for any physical bottlenecks(see section on Resource Monitor).6
NetworkAll communication between Sharegate and both theHaving sufficient upstream bandwidth to upload allsource and destination farms is done over HTTP/S andyour content will help you respect your desired timeas such, is done over your TCP/IP network. This meansframe. Also, factor in some overhead for the protocolthat it is subject to any bottlenecks you may have inas well as any other HTTP/S calls required to migrate.your network infrastructure such as network congestion.This can add up fairly quickly especially if you areThis can lead to packet loss as well as high latencymigrating a lot of structure elements.as well as bandwidth limitations.Example of speed when moving a 1gb fileBandwidth is typically not an issuewithin a LAN but can be an issuewhen migrating to Office 365since the migration will needto travel on the public internet.7
If you suspect you have network issues, check forthese elements:Check router/switch for elevated CPU/RAM usage. (This can lead to dropped packets.)Perform a continuous network ping test between the Sharegate workstation and your WFE. (Checkfor jitter, high latency, dropped packets.)Check with your ISP to make sure you have sufficient upstream bandwidth if you are migrating to Office 365. If you are using a VPN to access your farms, there may be additional latency added. 8Use a debugging tool such as Fiddler (or take a lookat our built-in Diagnostic Mode) to inspect the trafficbetween Sharegate and your SharePoint farms.If you are using a Proxy to communicate with yourIt can help you identify which calls are taking thefarms, check the Proxy for elevated CPU/RAMlongest to return and help you identify what partusage since this can lead to dropped packets.of your infrastructure is slowing the calls.
Insane Mode to Office 365Sharegate leverage Microsoft's Office 365 Import APIWhen importing to Office 365, look out for these items:to help our customer reach record migration speeds.Migrations that used to be limited to around 500 MB/h There is a limited number of import bots availableper Content Database in Office 365. There can onlycould now reach speeds as high as 35 GB/h with typicalbe a certain number of simultaneous import jobsusers seeing 5-15 GB/h.to each separate Content Database. If you planThis API relies on a set of services and infrastructureon importing a large amount of data (1 TB ) pleasewith their own bottlenecks and limitations. The mostcontact Office 365 support to provision the requiredprominent change is the need to stage all of theContent Databases ahead of time in order to spreadmigration content in an Azure Storage Accountout the load.for asynchronous ingestion by the Office 365 farm. If you opt to use a custom Azure Storage Accountwith the Insane Mode instead of the defaultcontainers, you should consider the data centerin which you create the account. Your Office 365GB/Htenant is located in a specific Azure data centerand uploading to a storage account found in anothergeographic location will add additional delaysto the import process. Using the default containerwill always ensure that the storage account is in thesame data center so this consideration is removed.9
Using encryption at rest for the contents foundin the Azure Storage Account can slow down theimport process by about 10%. The added securityis, in most situations, entirely justified but there canstill be a performance gain if you opt to not useencryption. If you are using the default storageaccount, encryption cannot be disabled. There are certain service limits built into AzureStorage Accounts. These are fairly generous and willprobably never be hit in a typical migration scenario(10Gbps maximum ingress / 200,000 IOPS). When uploading content to an Azure StorageAccount, you may run into bandwidth issuesdepending on the speed provided by your ISP.If you require additional speed when uploadingto Azure, you can look into creating an Azure ExpressRoute. This creates a fast, reliable and privateconnection directly between your data centerand Azure.10
Resource MonitorAs with all software that runs on a computer, performance will always be limited by the physical components thatthe computer is built from. In the case of a VM, the limit will depend on the specified settings for the VM as wellas the hardware powering the host. While Sharegate does not typically add an unmanageable stress on thesecomponents, in some cases servers or workstations that are already under substantial load can become sluggishor unresponsive.Here are a few indicators that you should lookfor in the Windows Resource Monitor:CPU Number of cores: Additional cores can helpspread out execution and allow additionalinstructions to be executed simultaneously. CPU - Total: If this is consistently high (over 80%)then you might run into a bottleneck Try to seeif you can reduce the number of active processeson the system.11
Memory DiskUsed Physical Memory: If this value is high, it means that processes will need to use virtualbetween a request to read or write (I/O) somethingmemory in order to store state and instructions.on disk and the start of the operation. A high queueAttempting to read a value that is in virtualtime means that there are a lot of processesmemory but not physical memory causes a Hardrequesting different elements on the disk at theFault (see below), which can significantly impactsame time, adding significant latency to operations.performance. The easiest solution here isThere are only two ways to reduce this delay. Youto increase physical memory through additionalcould change to a higher performing disk (eitherresources either in the VM settings for a VM,SSD, NAS, RamDisk, etc.) or reduce the number of I/or through added memory sticks for a physicalO requests. Hard Faults are also a significantserver or workstation. Queue Length: This represents the time it takescontributing factor to disk queue length.Hard Faults/sec: Hard Faults occur when a processattempts to read a memory address that is notcurrently loaded in physical memory. Windowsmust then recover the memory block from thepage file (stored on disk), and copy it to physicalmemory before execution can continue. This addssignificant latency to an operation and is a commoncause of application slowdown. If you see a lotof Hard Faults/sec, then increasing availablePhysical Memory can be an easy fix.12
ConclusionAbout SharegateThere are a lot of components at play whenSharegate helps thousands of IT professionalsperforming a SharePoint migration that might notworldwide manage, migrate and secure theirbe obvious at first. While in most scenarios theSharePoint & Office 365 environments. A productinfrastructure in place is more than capable of dealingmade with love by Montreal-based softwarewith the migration project, in cases where the desireddevelopment firm GSOFT, where we truly believe thatspeed is not obtained it is useful to know wheresimplicity and happiness are key to success!to look in order to find the bottleneck.There will always be a limiting factor to allmigrations, the idea is simply to adjust the differentcomponents until you reach a speed you areWant to learn more?Connect with us on Twitter and visit share-gate.comfor more SharePoint-related content.comfortable with and that matches your schedule.@sharegatetoolswww.share-gate.com
migration content in an Azure Storage Account for asynchronous ingestion by the Office 365 farm. When importing to Office 365, look out for these items: There is a limited number of import bots available per Content Database in Office 365. There can only be a certain number of simultaneous import jobs to each separate Content Database.
