SCOM behavior with Entities in DB and DW when deleting and re-importing MPs

I just want to share this piece of information because it’s not really documented and someone might wonder about how SCOM behaves with discovered entities in the Operations Manager DB and Data Warehouse DB. We had a discussion with our Microsoft contacts and here’s the (somewhat surprising) outcome.

Let’s start with a short overview, why this information is so important for us (and maybe for you too):

As many of you do, we deploy our own authored management packs containing our own class definitions which are discovered and monitored. In our case, we created a sealed MP, a “library” MP containing our classes, rules, monitors, etc. We also create MPs programmatically which have a reference to this library MP. Sometimes we are forced to update this library MP to a new version and sometimes it happens that we break compatibility so that we cannot just “upgrade” our MP. We are forced to uninstall all referenced MPs, the library MP and reinstall the new library MP (my friend Tenchuu wrote an awesome script to automate this process:http://systemcentercentral.com/BlogDetails/tabid/143/IndexId/55738/Default.aspx).

First question came up: what happens to my discovered entities when I uninstall the MP containing the discovery and class definitions?

The second question was: what happens when I re-import the MPs, the discovery runs and the exact same entities with the same ID are discovered again?

The answer to the first question was – as expected:

When you uninstall the MP containing the class definitions and discovery, all discovered entities are deleted and all associated operational data (Events, Alerts, Performance Date, State Change Data, etc.) will be gone as well.

So everything is gone from the OperationsManager database.

In the data warehouse database the data will still be there.

The answer to the second question may be obvious as well:

When you reinstall your MPs and the discovery will create your entities with the same object IDs as before, the data warehouse would continue to associate the data to the same object you had before. Except for the missing data while you had your MP deleted, you will not see a difference.

There’s one caveat and you should be very careful: Since the data warehouse also keeps track of the MP version with each object ID, the data warehouse will only keep the data from the last three MP versions in the database. This is not configurable! In other words, when you keep your data warehouse data for 6 months and you do uninstalling and re-importing your MPs more than 3 times in one month, you will lose more than 5 months of data.

UPDATE: The above – the rule of 3 – only applies to unsealed Management Packs. When you use sealed Management Packs the DW data will always be kept until standard grooming kicks it out, regardless how often you update, delete or re-import the MPs.
One other small detail is, that you cannot trick this mechanism by just not changing the MP version. The data warehouse will create an “internal” versioning of the MPs which will change upon update, even if the MP version is still the same!