Fixing Information Store Error -2147221213 (ecTimeSkew) – Exchange 2007 / 2010 / 2013

(Late) Sunday nights are not a fun time to get paged about a customer down…

Situation

Engineer reboots an Exchange 2010 server that is apart of a two node DAG, but when the server comes back online the MS Exchange Information Store service is stuck in the “Starting” phase within the Services MMC, and the Microsoft Exchange System Attendant will just not start. All other services are online. What gives?

Environment

Another pretty simple buildout, Exchange 2010 SP3UR6 running on top of Server 2008 R2 with all patches (security / hotfixes) that are available. Both servers are multiple-role (CAS/HUB/Mailbox) which also has a CAS array and a two node DAG. Two copies of each database in the environment.

Troubleshooting

The first thing I did was confirm that the only two services that were boinked at the time were the Information Store and System Attendant. Best way to actually check this is by running Test-ServiceHealth if you are running Exchange 2010 / 2013, which showed exactly that.

When I looked in the Services MMC I did see the Exchange Information Store stuck in the starting state. I looked in the Event Viewer: Application logs to find this error:

Log Name:      Application
Source:        MSExchangeIS
Date:          8/24/2014 11:51:29 PM
Event ID:      5003
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCH01.company.com
Description:
Unable to initialize the Information Store service because the clocks on the client and server are skewed. This may be caused by a time change either on the client or on the server, and may require a restart of that computer. Verify that your domain is correctly configured and  is currently online.

The first thing I did was make sure the Windows Time service was properly pulling the right time source..

w32tm /query /source

Since that looked proper (it was the PDC) I moved on and physically checked the time between the two Exchange servers, which looked fine. Confused, I decided to try a force sync of the Windows Time service..

w32tm /resync

Afterwards, performed a reboot and avail.. nothing. Back to square one with the databases showing a state of “Failed” and but this time the Information Store service did not get stuck in “starting”, it was stopped all together. I attempted to start the service with the following error message:

“Unable to start service. Error -2147221213. Please contact your support provider or the vendor”

This is a hex error. Luckily enough if you know the internals a bit to Exchange, sometimes you can look up these errors using the Err.exe tool MSFT provides (link).

That then shows us this..

# for decimal -2147221213 / hex 0x80040123 :
  ecTimeSkew                                                    ec.h
  MAPI_E_INVALID_ACCESS_TIME                                    mapicode.h

We now know its a windows time issue, so now what?

Resolution

There are two routes you can go with this:

– Manually set the Windows Time service to sync off the PDC (which is what I did)

– Completely reset the Windows Time service

Manually set Windows Time service

This is pretty easy, and all you have to do is run this command

net time \\PDCName /Set

It will ask you if you want to confirm, click Y and then confirm that the time is now synced. Afterwards, restart the MS Exchange Active Directory Topology service which will restart all the other Exchange services, and hopefully its fixed for you.

If that doesn’t work try resetting Windows Time all together, and then set it to point to the PDC (sometimes required)..

Reset the Windows Time service

Open up a command prompt and then each of the commands below (each line break is a new command to run)..

net stop w32time
w32tm /unregister
w32tm /register
net start w32time

Once this is done, you should check the actual provider to make sure it is either the DC that is hosting the PDC FSMO role (recommended) or another domain controller that is local to Exchange (e.g: within the same AD site)

w32tm /query /source

If it is NOT the DC holding the PDC FSMO role or even a DC (it might show as “CMOS”, which means your BIOS) then I would set it to either one of the options I said above..

net time \\PDCName /Set

At this point restart the MS Exchange Active Directory Topology service and make sure everything starts.

Once either one of these steps are done, you should be fully working again.

 

TL;DR: Windows Time sync is required for domain controller authentication on a domain joined machine, along with authentication requests. If windows time breaks (or Windows thinks its broken when the time is actually synced), now you know how to fix it!

Leave any questions and comments below.

 

Cheers!

 

– Adam F

 

 

 

 

 

 

 

Advertisements

5 thoughts on “Fixing Information Store Error -2147221213 (ecTimeSkew) – Exchange 2007 / 2010 / 2013

  1. Thank you a lot!!
    You saved my time really!!!
    Thanks’ for sharing this great info.
    If I may help you with anything feel free to ask.

    Thanks’ again :-).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s