So I have something weird going on and i can not pin down exactly what is causing it. My asp.net project is live with session state on two production servers that are synced using the following command:
msdeploy -verb:sync -source:webserver,computername=%MACHINE%,username=Administrator,password=%PASSWORD% -dest:webserver 2<&1
The application is an asp.net 4.0 website that is run on two Server 2008 R2 web servers behind a load balanced configuration where the users are set to stick to one server once they connect. We have <MachineKey>
set hardcoded with validation and decryption keys in the root site of the application and it is the same between both servers. My application is set up to forward exception events to our email system.
What is happening is that i am receiving the dreaded 'Validation of viewstate MAC failed' from the servers but even though the server load is 50/50 split the errors are coming in on a 99/1 split. So one web server is generating these errors considerably more often than the other one. This is strange considering the servers are synced and all configurations are identical.
I've done extensive searching on this problem and it seems quite difficult to find any solution that doesn't mention or do the following.
<MachineKey>
is not identical between servers. (I know for a fact this is not my problem)
- Setting
enableViewStateMac=false
or some other setting that jeopardizes the site security.
- Make sure that all action tags on form inputs reference the same page they are placed on
- Make sure the instance ID of the servers are the same (they are)
- If the user clicks through the page before the entire page (viewstate) has been downloaded (my viewstate is set to render at the top of the page).
- Issues with response.redirect and server.transfer
Now i have eliminated all except the last two as possible causes. My application has been running fine for over a year with no issues and right before these errors appeared i enabled SQL session state, migrated the project from .NET 3.5 to .NET 4.0, and set the set the server mode deployment mode to retail. I have tried recycling the application pools and performing an 'iis reset' to no avail.
Does anyone else have any suggestions as to what i can look at? Bottom line i do NOT want to fix this by opening up security holes in my site.
It appears this is happening to users right after they authenticate using forms authentication the first time they try to log in but i can not confirm this. I also have a theory that this might have to do with caching but i can't be sure on this either.
Here is the juicy bit from my web.config (i have removed some sensitive information)
<system.web>
<httpRuntime requestValidationMode="2.0"/>
<globalization culture="en-US" uiCulture="en-US" resourceProviderFactoryType="WebResourceFactory"/>
<compilation debug="true" defaultLanguage="c#" explicit="true" strict="true" targetFramework="4.0">
<assemblies>
</assemblies>
</compilation>
<authentication mode="Forms">
<forms name=".ASPXAUTH" loginUrl="Login.aspx" protection="All" slidingExpiration="true"/>
</authentication>
<authorization>
<deny users="?"/>
</authorization>
<sessionState mode="SQLServer" sqlConnectionString="connection" compressionEnabled="true" />
<pages theme="Blue" controlRenderingCompatibilityVersion="3.5" clientIDMode="AutoID">
<machineKey validationKey="key" decryptionKey="key" decryption="3DES" validation="SHA1" />
</system.web>
EDIT:Emphasized that i'm using SQL session state with a load balancer set to prefer to route users to the server they started on.