2009/08/21

Fixing “Padding is invalid and cannot be removed” when requesting WebResource.axd

If you are using ASP.NET in your website and have a look at your Application EventLog you will probably see warning entries like this:

CryptographicException: Padding is invalid and cannot be removed.

Event Type: Warning
Event Source: ASP.NET 2.0.50727.0
Event Category: Web Event
Event ID: 1309
Date:  21/08/2009
Time:  13:08:48
User:  N/A
Equipo: WEBSERVER
Description:
  Event code: 3005
  Event message: An unhandled exception has occurred.
  Event time: 21/08/2009 13:08:48
  Event time (UTC): 21/08/2009 11:08:48
  Event ID: 1cc59501bae34562a1e486c16f2e799f
  Event sequence: 11912
  Event occurrence: 1
  Event detail code: 0
  Application information:
    Application domain: /LM/W3SVC/1/ROOT-1-128952696565995867
    Trust level: Full
    Application Virtual Path: /
    Application Path: C:\Inetpub\webs\www.test-domain.com\
   Machine name: WEBSERVER
  Process information:
    Process ID: 3920
    Process name: w3wp.exe
    Account name: TEST-DOMAIN\IWAM_WEBSERVER
  Exception information:
    Exception type: CryptographicException
    Exception message: Padding is invalid and cannot be removed.
  Request information:
    Request URL: http://www.test-domain.com/WebResource.axd?d=pFeBotgPWN6u7M4UfAnWTw2&t=633687432177195930
    Request path: /WebResource.axd
    User host address: 127.0.0.1
    User:
     Is authenticated: False
    Authentication Type:
     Thread account name: TEST-DOMAIN\IWAM_WEBSERVER
  Thread information:
    Thread ID: 12
    Thread account name: TEST-DOMAIN\IWAM_WEBSERVER
    Is impersonating: False
    Stack trace:
       at System.Security.Cryptography.RijndaelManagedTransform.DecryptData(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Byte[]& outputBuffer, Int32 outputOffset, PaddingMode paddingMode, Boolean fLast)
       at System.Security.Cryptography.RijndaelManagedTransform.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
       at System.Security.Cryptography.CryptoStream.FlushFinalBlock()
       at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, IVType ivType, Boolean useValidationSymAlgo)
       at System.Web.UI.Page.DecryptStringWithIV(String s, IVType ivType)
       at System.Web.Handlers.AssemblyResourceLoader.System.Web.IHttpHandler.ProcessRequest(HttpContext context)
       at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
       at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Custom event details:
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Depending on how busy is your web server you can see them appear from time to time or up to every few minutes, thus filling your EventLog and being from a light annoyance up to a real problem (depending on how hypochondriac you are).

In fact, they are just warnings that can be ignored on most of the cases, but they can be a real problem when they bury other events and the forest do not let you see the trees. If there are many of them and you want to get rid of them (or most of them at least), keep on reading.

You might check your IIS Log by the times when the warnings appear and (if you also log user-agent) you will probably see that most of the time the URL is NOT requested by a real user, but a spider engine doing its crawl (googlebot, msnbot, yahoo, tahoma, or any other). You can double check doing a reverse dns check for the offending IP address doing a ping –a aaa.bbb.ccc.ddd and you will also see the IP resolves to something like *.googlebot.com, *.search.msn.com, *.crawl.yahoo.net or *.ask.com. This should give you a hint on what to do…

WebResource.axd is just an httpHandler that wraps several resources within the same DLL. It is in charge of returning from little .gif files for serving the arrows of the ASP:Menu control, to .js files governing the behavior of the menu itself. Even though your website do not use ASP:Menu control, you probably will be using WebResource.axd for javascript dealing the post back of your form or any other thing.

Why does this exception happen?

If you see in detail the parameters following the WebResource.axd request you will notice two of them. The first one d refers to a particular resource embedded in the httpHandler DLL. It is a fixed value as long as the source DLL is not updated or recompiled. The second t parameter is a timestamp parameter that changes whenever the web application (AppPool) is recompiled (a changed/updated DLL, an update to web.config, and so) and depends on the machineKey of the web site. If web.config does not explicitly declare a fixed machineKey, the t parameter will change from time to time (restarts, job recycles, etc).

In fact these CryptographicException warnings are well known in web farms configurations. In that case, all the servers belonging to the same farm must have the same machineKey because if a served page (.aspx container page) by a particular server of the farm includes a value of t parameter and the subsequent request for that URL resource is handled by other server of the farm, the exception would arise and the user could not download the resource. And, in this case we would be talking about real browsers with real users behind them, not spider engines.

Furthermore, if you have implemented a conditional GET in your webserver, this exception is more likely to happen, since a user can come back to your site, do a request for a page that has not changed, being returned a 304 Not Modified, and still request the resources included in that page, that might be invalid due to the change of t.

The solution: two steps.

As you can imagine, the first thing that you can do is setting a fixed machineKey in your web.config file. Even though you are not running a cluster, nor a web farm, it will help you to minimize the occurrences of the warning Padding is invalid and cannot be removed.

For this you can use a machineKey generator or generate your own if you know how to do it (random chars will not work).

<system.web>
  <machineKey
    validationKey='A06BDCF2F6CF.A.VERY.LONG.44F13E76184945A7C477601'
    decryptionKey='99079B21C2F3644.A.BIT.SHORTER.BB81C7E9D58378'
    validation='SHA1'/>
</system.web>

The second (and easier) step to follow is to prevent WebResource.axd URLs from being requested as much as possible. In particular by search engines crawlers or bots, since those resources should not be indexed nor cached in any way by them. Those URLs are not real content to be indexed. If you only add the following lines to your robots.txt you will see how the frequency of CryptographicException is reduced drastically. If you also change the machineKey to a static value, you will get rid of them almost completely.

User-agent: *
Disallow: /WebResource.axd

As I said, you will get rid of this warning almost completely. There might be search engines not following your robots.txt policies, users visiting you from a Google cached page version, etc. so you cannot get rid of this warning messages for good, but yet enough for not being a problem anymore.

Summary.

Summing up, this event appears when there is a big time difference (lap time) between the page that contains the resource and the resource itself being requested. During that lapse, the application pool might have been recycled, recompiled, the server restarted, etc, thus changing the value of t and thus, rendering the older t value useless (the cryptographic checks fail).

Links.

Internet Information Services IIS optimization

Keywords.

WebResource.axd, CryptographicException, padding, invalid, removed, machineKey, exception, warning, IIS

14 comments:

Anonymous said...

Thanks for this really informative article and the background information, especially the t parameter.

bizl said...

v. nice! works wonders for my event viewer

Vlad said...

it doesn't work for cached pages a couple with Opera.

José Antonio García Barceló said...

You will not be able to get rid of these warning messages completely, but you can certainly reduce them using the steps provided. Single warnings here and there from time to time are quite common, you should not worry about those.

greenstone90 said...

Excellent posting on this issue I am seeing also on my IIS server. I implemented both the suggestions (adding the machine key and the robots.txt), but unfortunately, I am still getting about ten to twenty of these messages a day (about the same rate as before). Thoughts on any other ways to resolve this error message?

Thanks!

José Antonio García Barceló said...

From my point of view, 10 to 20 warnings like this (they are not errors) are no issue, but if you want to track them down, I would suggest to have a look at your IIS logs at the time that the warnings appear in the EventLog, so you could find more about the browser doing the request: IP address, UserAgent, previous and subsequent requests, etc.

Good luck and thanks for your comment ;)

Rosebud said...

Looks like this fixed it for me. Nice article. Good job!

Anonymous said...

Thanks for sharing that. I almost blamed my VAIO P on Ubuntu. Good thing it was just a matter of web.config things

Anonymous said...

Excellent info José. Thanks for posting it up.

Zeeshan Umar said...

Really a nice article, it worked for me very good job.

Anonymous said...

This doesnot work in case of sub domains. Any ideas?

Kiran banda said...

Thank you so much for a detailed explanation.

Suyog Pathrikar said...

Thank you very much this really help us a lot

Anonymous said...

Really Nice articles, Save lots of times.

Thanks Man!!!