Considerations for Migration and Merge of Orgs - Lessons Learned

In a previous post we established a plan to do a whole org migration. We want to give you the real deal here without any oversimplification, and real life is not simple and plans never go as expected. Having now completed a couple of migration projects, there’s a few lessons and some adjustments we need to make clear for the record.

Packages Can Be Such A Drag

As we discussed in our previous post in the section “How to Move Metadata Between Production Environments”, packages and deployment API tools are the only possible ways to deploy metadata from one environment to another when the environments are not connected. Packages are a very practical declarative (point-and-click) way to put together everything you want to migrate from the source org and then install it in the destination org, BUT there are some very important caveats you must prepare for (and so it begins... a story full of buts).

Change sets (when deploying between connected environments) allow you to select one item of metadata, and only that one item is added to the change set upon selection. BUT packages work on automatic addition of dependencies, which means that when you add an item of metadata, it automatically adds (drags) into the package any metadata related to that item.

For example: Object A has a lookup field to Object B and you want to migrate Object A but not Object B. When building the package, adding Object A is going to automatically drag all its fields into the package, including the lookup field to Object B, and in chain, automatically drag Object B and all its fields into the package as well.

This is a major issue when there’s metadata you decide you don’t want to move to your new org. If packages drag stuff into them, it makes it impossible to separate our migration into different steps as we had initially planned:

• Migrate data model
• Migrate extended data model
• Migrate small functionality
• Migrate big functionality

One way or the other stuff gets added in chain to form one giant package. 

Our Three Package Options in Detail

Option 1: Delete the undesired metadata already from the source org, so that it doesn’t get dragged into the package in the first place. This sounds like a good option, BUT remember that while you are working on the migration to the destination org, the source org is the one that is live to users. So doing such a clean up could disrupt their work. In addition, you may want to keep all that metadata in the source org for as long as possible as your backup plan, until your destination org is actually ready to go live. So there’s already two good reasons not to pursue this option.

Option 2: Package everything you want to migrate, let it drag the dependencies (even the undesired ones) and install the package in the destination org. After installation of the package you can delete whatever undesired metadata was dragged into the package in the destination org while your source org remains untouched. This is some manual work, but depending on how your entire org is built, it will likely be less work than option 1 (likely there’s less undesired metadata that gets dragged into the package than the overall undesired metadata).

Option 3: For some metadata (likely not all) that particularly drags a lot of undesired metadata, you may want a different approach. For example: let’s say you want to push a report that drags fields you don’t want to migrate. Instead of putting it in the package dragging a lot of undesired metadata, you can use a deployment API tool to download the metadata from the source org to your desktop, Edit the metadata to remove the references to the undesired metadata, and upload the modified metadata to your destination org.

This way you get to keep your source org untouched and you don’t drag any undesired metadata to your destination org at all. You could potentially consider this approach for the entire migration and not use packages at all, BUT keep in mind the following. 

This would require someone with some development skills to spot exactly the places where the metadata needs to be removed. Not any removal of specific items done this way is guaranteed to work, as Salesforce has some restrictions on how much you can change something that’s already created in a certain way (for example: you cannot change a report type for a report in most cases).

Doing this kind of editing in a few metadata items is one thing, BUT if you have to do it for dozens, hundreds, maybe even thousands of list views, report types, and reports, then it’s no longer something worth investing so much time in. If you have to delete 20 fields from one report then it would be less work to do this approach. If you have to delete 20 fields from 20 reports then probably option 2 would take less work.

Our Three Options Overall

So overall it seems like option 2 is most viable for most migration cases, complimented by option 3 in some very specific punctual cases. BUT we’re not done with the buts just yet.

Managed vs Unmanaged Packages 

When it comes to packages, not all packages are upgradable (meaning you can install one version of the package first and then another version with additional components afterwards - which was the purpose of our initial plan). There are two types of packages: unmanaged and managed. 

The unmanaged type allows for the components to be edited after installation in the destination org, which is something we definitely want, we need the installed metadata to be editable in our new org so that we can continue building on top of it in the future. BUT unmanaged packages are not upgradable. This means they could be used for migration, but it has to be a one-time all-in-one package.

Let’s say that you are able to package (unmanaged) and migrate one object (with its fields) and only that object (because it’s an independent object that is not connected to any other so nothing else is dragged into the package). 

If you want to create a second version of that package or even another unmanaged package to migrate for example a process builder on that object, the object (and its fields) would be once again dragged into the package. If you try to use this second package to install it in an org where the first package is already installed, it won’t allow you to. It will say that the object already exists with that name and it won’t try to overwrite it or consider it the same:

When you try to install it may give you the chance to automatically rename the conflicting components with the second option “Rename conflicting components in package” as shown below.

BUT that doesn’t work for objects or fields. Even if it worked, you would end up with two objects in the destination org instead of one, which is not the purpose either. So this leaves us with only one option:

We migrate all at once in one unmanaged package and then in the destination org after installation remove any undesired metadata that was dragged but not meant to be kept. If we miss something in the package, we can add the missing component, create a second version of the package, uninstall the first version of the package and then install the second one (to avoid the conflict explained above). This is troublesome, because what if you realize you missed a component way after you already started making changes to the first version installed (like the removal of the undesired metadata)? Uninstalling means you would have to make those changes all over again on the second version upon installation. Basically it’s like you get one shot, and any other shot would add a lot of work that will have to be done twice over a small omission.

Managed packages are mostly intended for apps to be published in Appexchange, not migrations. They are upgradable, BUT when you put something into a managed package and the package is installed, any code and some metadata is not visible in the installation org for later editing. This does not meet our expectations at all, as we want to be able to keep building in the future on top of the items migrated as we would in the original org.

Besides, not all orgs are allowed to create managed packages, so this may not even be an option for most. If you go to Setup > Packaging > Package Manager in your org you may find the following message:

Lastly, as we mentioned in the original post some metadata is not packageable. This includes anything depending on the users and organization:

• Profiles
• Roles
• Approval Processes: if a step is assigned to a particular user or queue, you may have to temporarily assign the step to the current user or remove the step overall, then deploy it to the destination org and then manually restore the step with the user as it’s created in the destination org.
• Public Groups
• Queues
• Personal (private) List Views, Reports or Dashboards

So there will be some manual work to be done anyways no matter which approach you use.

Package Options Conclusion

The best way to do the migration is with a package that is unmanaged and unique.

Unmanaged: because we want all metadata to be editable in the destination org and it’s likely the only type of package you’ll be able to create for your org type.

Unique: because managed packages cannot be upgraded, so we’ll have to build one with everything we want to migrate (even the undesired metadata dependencies).

Install the package in the destination org then delete the undesired metadata in the destination org. And for some particular items do either a manual migration or migration via API tool.

Forgot Something In Your Package? We’ve Got You Covered.

Let’s say you are testing your migration package in a sandbox environment of your destination org (sandbox 1). The sandbox has the first version of the migration package installed for testing before you install it in production. The destination production org doesn’t have the package installed yet (because you installed it in sandbox 1 to test it out first). You already spent time and effort in migrating data to sandbox 1 so that users could test the overall migration.

BUT you just noticed you forgot to include a couple of features in your package (features A, B, C). These cannot be packaged separately because packages drag related metadata (you cannot isolate particular features of metadata in a package), so it should be included in a new version of the package (version 2). 

As we discussed before, we can’t install a new version of the package on top of the old one in sandbox 1 because this is an unmanaged package restriction, and if you uninstall version 1 of the package in sandbox 1 to install version 2, you will lose all the data you already migrated and that power users already tested with. 

What can you do?

1. Add the missing features to the package in the source production org and create a new version of the package (version 2).
1. This version will contain everything from version 1 plus the new features: Package version 2 = Package version 1 + Features A, B, C
2. Create a new sandbox (sandbox 2) from the destination production org.
3. The destination production org doesn’t have the old package installed yet, so you will be able to install version 2 of the package in sandbox 2.
4. Create a change set to move the features A, B, C from sandbox 2 to sandbox 1. Since these sandboxes are from the same production environment, they can be connected to do deployments from and to one another.

This way you can have in sandbox 1 the same effect as if you had installed version 2 of the package without disrupting or undoing the data loading and testing already done.

The Migration Plan Reviewed

So “The Migration Plan” in our previous post needed to be shuffled a bit in order to accommodate the steps given the technical restrictions, while minimizing the manual removal of dependencies before and after deploying to the destination org.

The edited plan and summary of the new plan are outlined here in a doc you can copy and work from yourself.  In summary, here’s the cleaned-up version of the new plan:

1. Install Appexchange packages without configuring them. See data migration (Step 8) as to why not configuring them yet.
2. Prepare the unmanaged package in the source org and install it in the destination org to:
1. Migrate data model (objects and fields and record types).
2. Migrate extended data model (validation rules, page layouts, lightning record pages, compact layouts, actions, buttons, matching rules, duplicate rules, list views, tabs, apps, homepage layouts, folders, report types, reports, dashboards, DLRS, static resources, custom settings, custom metadata types, everything else that is not automation)
3. Migrate small functionality (declarative automation: workflow rules, process builders, flows, email alerts, email templates, field updates, etc).
1. Deactivate all validation rules, workflow rules, process builders, flows, etc. so that they don’t affect the data migration later.
4. Migrate big functionality (Apex classes, Visualforce pages, Apex triggers, Lightning components).
1. You might need to update your test methods in order to make the code deployable if your test classes are not currently passing.
2. Deactivate all triggers so that they don’t affect the data load later.
3. Remove from the destination org any undesired metadata that was dragged from the previous migration steps.
4. Migrate users (activated but not with the set password email sent out).
5. Migrate anything that could not be migrated with packages because they are dependant on users:
1. Profiles
2. Roles
3. Approval Processes
4. Public Groups
5. Queues
6. Personal (private) List Views, Reports or Dashboards
6. Restore anything that was momentarily changed to be packaged:
1. Dashboards
2. Email Alerts
7. Migrate MyDomain:
1. Make sure that all references are switched to the new domain when migrating. See more information here Update Your Salesforce Org for Your New Domain.
8. Migrate data:
1. From step 2 we have any functionality inactive so that it does not interfere with the data migration.
9. Configure Appexchange packages installed in step 1.
10. Configure Security (profiles, roles, permission sets)
11. Reactivate and restore any functionality that was previously migrated inactive.
12. Regression testing:
1. Make sure all features are working as expected with data entered from now on.
2. This is to guarantee all desired metadata has been migrated and all conflicting processes are resolved.
3. This should include the team of stakeholders that decided which processes would make it into the new org.
13. Freeze old org:
1. Disable users in old org. The old org will no longer be accessible to end users other than System Administrators so that the new org can go live.
2. Migrate any data that was input (created or modified) in the old org between data migration (Step 8) and now to the new org.
14. Enable users in new org:
1. Now all users will be able to set their password and log in to the new environment with all data and the features selected as final.
15. Connect any 3rd party integrations to your new org. Update the credentials, certificates or anything used by an external system to connect to Salesforce to point to your new production environment.
16. Retire old org:
1. Now the old org can be let go.
2. All metadata and data should be backed up before this org is completely disabled by Salesforce.

The Takeaway

Plans are always designed to the best of your knowledge at the time. But they should also be flexible to accommodate the unexpected, the drawbacks, the delays, the additional work that may pop out of nowhere. Part of that flexibility is to accommodate the steps of your process depending on the specific needs of the project to minimize any manual work. Give yourself a lot of time for analysis before you start to make the best plan possible, so that you can save more time when you actually get to do it.

Resources

Here are some resources to learn more:

• Work with Packages

What do you think of this summary? Is there anything else that should be brought to attention when migrating/merging orgs? Tell me all about it in the comments below, in the Salesforce Trailblazer Community, or tweet directly at me @mdigenioarkus. Subscribe to the Arkus Newsletter here to get the top posts of the Arkus blog directly to your inbox.