How to merge MailStore archives

If you have recently upgraded to MailStore version 9 you may not be aware that the new maximum number of messages that an archive can contain has now been increased from 500,000 to  5,000,000 messages.

If you are like me and you have found yourself with a collection of much smaller MailStore message archives it is now much easier to merge these together into a smaller number of larger archives.

Let’s Begin

This process can take some time but during the process, any messages that are duplicated across archives will get de-duplicated so you may find that the overall storage required drops.

You can also find that in deployments that have 100’s of archive stores reducing this to 10 or less will dramatically increase archive throughput and will vastly improve users experience.

To merge archives within MailStore we first need to know the Archive ID which we can find in the Storage location window.

(Administrative tools -> Storage -> Storage Locations)

In this example, I am going to merge the top two archives ‘2009-12’ and ‘2010-11’

Management Shell commands

Now we know which archives we wish to merge we need to open the Management Shell (Administrative tools -> Management API > Command Prompt)  and run the command..

MergeStore -id=29 -sourceId=40 

(Please note this command is case sensitive ‘sourceId’)

If you have issues with the above command try..

MergeStore –id=29 –sourceId=40 

In this example, I want to merge message Archive ID 40 into Archive with ID 29. This will copy the message from Archive ID 40 and store them into Archive ID 29, so at the end, we should end up with an Archive with 1,027,103 messages in it.

During the merge, you should see the number of messages that have been completed. Please be aware this will take a long time when dealing with 1000’s of messages!

In my testing, I was seeing about 30 messages a second being merged so 500,000 messages would take about 4.5 hours to merge.

Once finished if we now look at the storage locations you can see Archive store ’29’ now contains all of the messages from Archive store ’40’ and Archive 40 is still attached.

We no longer need the old Archive store 40 and in effect now contains duplicates only, so we can detach this old store using either the following command in the Management Shell window.

DetatchStore 40

…or manually by right-clicking on it and choosing ‘detach’.

So there we have it in its simplest form we have merged two archives.

Don’t forget to delete the source files

Now you have detached the message store you are safe to delete the archive store folder to free up drive space. In our example, the old message store was located in D:\mailarchive and was named ‘2010-11’. Make sure you delete the correct store if in doubt rename it first and make sure you can still see all the archives you are using.

Those of you that have a large number of archives may have noticed that with a little pre-planning it would be possible to script this process to add multiple archive stores together and detach the old ones once that has been done.

I won’t cover this more advanced method in this guide, but if you do have a large number of archive stores you need to merge and you want to automate the process please drop me a comment below.

10 thoughts on “How to merge MailStore archives

  1. I can’t do it in Mailstore home, since the home version doesn’t have storage in the admin tool box, and therefore your recommandation:
    Storage -> Storage Locations>
    cannot be performed.

    • No, That’s correct this is in respect to MailStore Server product, The home product is a free cut down version of the product that does not use multiple storage locations like the server product.

    • Hi Amk, Can I ask what version of MailStore you are running as you should only need the single “-” of the current version 11.2.0

      • Hi, I’m using version 10, also typing the command rather than copy/pasting as browsers and email clients sometimes change punctuation to “friendly” versions (replacing a dash with an en-dash or em-dash).

      • Hi Amk, Ok yes this command was different in Version 10. I have asked for the developers to update the built-in documentation when you type ‘help’ in the management API window, however, this would only affect the current release.

  2. Any thoughts on the following errors? Last datastore (around 120.000 messages) had 33 of them:

    An error occurred while merging message 43151 from store 21 to store 34: The message could not be entirely retrieved from the archive.
    An error occurred while merging message 43152 from store 21 to store 34: The message could not be entirely retrieved from the archive.

    Can I retry these specific messages, or what happens when I try to merge the entire datastore again?

    • Hi Axel,

      this error would usually imply that the source store has errors and some of the messages cannot be retrieved. If you first run a Check Data Integrity on that source store you should see if there are any errors reported. You may find you have damaged .dat files that need to be recovered from backups first before running the merge.

      • Source store verification gave some errors, so that explains the errors while merging. Can I re-run the merge to also merge the messages that could not be merged the first time, or would that create a lot of duplicates?

      • You can still run the merge it will just have to skip messages it cannot access so the final merge will be fewer messages. We sometimes have to do this as a way of rescuing good messages out of a slightly damaged archive store data set.

Let us know what you think....

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s