I had the following erroneous code in a SQL Server stored C# procedure:
class P
{
private DateTime? e;
public P(
DateTime? e)
{
this.e= e;
}
public DateTime? E
{
get
{
return E; // correction return e;
}
set
{
E= value; // correction e = value;
}
}
}
Calling the getter of E of the class P creates an infinite number of method calls. This causes the .NET stack overflow. This sometimes caused our SQL crash. Here's the log:
29.3.2007 9:46:08 A fatal error occurred in .NET Framework runtime. The server is shutting down.
29.3.2007 9:46:10 Microsoft SQL Server 2005 - 9.00.2153.00 (X64)
May 9 2006 13:58:37
Copyright (c) 1988-2005 Microsoft Corporation
In my opinion an error in SQL server CLR stored procedure must not be able to crash the whole SQL Server, as it seems to do. Could someone verify this?
What makes finding problems like this problematic is that the only error is like ".NET stack overflow". No pointer to where in the code the error occured. It took hours for me to find the problem
Hi JM_F,
This type of coding error certainly should not cause a shutdown in SQL Server, as SQL should be resiliant to stack overflow issues within the CLR. When I run your repro on my machine (x86) I get the expected behavior:
Msg 6538, Level 16, State 49, Procedure sp, Line 0
.NET Framework execution was aborted because of stack overflow.
It's likely that you are hitting a bug in the CLR. Can you describe your specific repro scenario that causes SQL to crash? You mention that SQL only crashes sometimes when you call this SP, do you know of anything that is different between when it causes SQL to crash, and when it does not?
Can you file this as a bug on connect.microsoft.com with any extra information you have? This will make it easier to investigate. If there is a dump file located in your MSSQL\LOG directory, please include that in the bug or email them to me directly at stevehem at microsoft dot com.
Thanks,
Steven
|||
Thank you for your quick reply.
We have two SQL servers installed on two physical servers.
A: a 64-bit Windows server. SQL Server 2005 SP1. The production server.
B: a 64-bit Windows server. SQL Server 2005 SP1. The test server.
The problem in the C# code causes the error message you sent when I run it on B. I've never managed to reproduce the crash on B. Because A is our production server, I can't try to reproduce the problem there anymore. Yesterday I run the erroneuos code maybe ten of times in A and it crashed two times (that I can see from the server log).
I'll send next week a message to connect.microsoft.com with more detailed information that I don't want to put here in the discussion forum.
I don't know whether this is related to the CLR problem, but yesterday evening and today morning the SQL server A got somehow jammed. For example, we couldn't open Management Studio and open Programmability/Stored procedures. It just didn't open. Also the CLR stored procedures just didn't work, they never seemed to finish. We had to restart the SQL server A (I guess this was done by starting and stopping the windows service, don't know for sure, because I didn't do this). Now we have a new problem with registering CLR stored procedures - both of the following commands produce an error:
CREATE PROCEDURE [dbo].prA AS EXTERNAL NAME a.xxxxxx;
GO
drop ASSEMBLY a;
Msg 6512, Level 16, State 27, Procedure prLoadOptionContracts, Line 1
Failed to initialize the Common Language Runtime (CLR) v2.0.50727 with HRESULT 0x80131522. You need to restart SQL server to use CLR integration features.
I guess we have to restart the server again?
|||
This is interesting - this error means that the SQL failed to bind to the CLR host on your server. This is usually due to a bad installation of either the CLR or SqlServer, but in very rare cases it can be a result of memory pressure on the machine when the CLR tries to load. So, if after the server restarts the CLR is able to load, that means it was a memory pressure problem, but if it persists then it is an installation one. The DMV sys.dm_clr_properties may also contain some additional information about what state the CLR is in. Is your server running under WOW?
I'm really not sure if these problems are related. You're the only customer I'm aware of that has hit either of them (a search engine check for both of your error messages returns this thread only). So either you are extremely unlucky or there is some underlying cause for the failures.
Steven
|||
Steven Hemingray - MSFT wrote: |
|
The DMV sys.dm_clr_properties may also contain some additional information about what state the CLR is in. |
dm_clr_properties actually gives information:
select * from sys.dm_clr_properties
name value
directory
version
state CLR initialization permanently failed
Steven Hemingray - MSFT wrote: |
|
Is your server running under WOW? |
I'm not the sysadmin / DBA so I don't actually know. Is there a simple way to check this?
The CLR stored procedures have been working well inside the SQL server, but there definitely has been other problems with the SQL server installation previously. Some of the issues still are unresolved.
I'll provide you with more details:
Last week, on the same day this "CLR crash" occured the server got somehow "stuck". This happened after these CLR problems. The symptoms were that if I tried to open stored procedures "leaf" in Management Studio, it just didn't open. Additionally the CLR stored procedures didn't seem to finish at all. They consumed CPU, but never finished (or at least we didn't want to wait > 15 minutes for a procedure that normally finishes in < one minute). We had to restart the SQL server procedure. After this restart I get this "you need to restart"-message.
There also are performance / availability problems with the server. I don't know the details of these problems, but I've heard from the others who are using the server that occasionally the server gets jammed "for a while" (15 minutes, let's say) and it doesn't seem to respond to anything. This issue is just for background information, I know there's not a lot you can do with the problem with this ambiguous problem statement. Everything just seems to point at there's something wrong with the SQL Server installation.
|||Today we restarted the SQL server. Now CLR gets initialied:
select * from sys.dm_clr_properties
directory C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\
version v2.0.50727
state CLR is initialized
I installed all the CLR stored procedures, log entry:
Common language runtime (CLR) functionality initialized using CLR version v2.0.50727 from C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\.
I tried to run one of the stored procedures -> SQL server crash with message:"A fatal error occurred in .NET Framework runtime. The server is shutting down."
I filed a bug report (266911) to connect.microsoft.com about the issue. It's private, since I uploaded the server log which may contain sensitive information. Let's hope I get some answers from there.|||
Thanks for filing the bug. Just to confirm, this stored procedure you run is still the erroneous stored procedure with the error causing stack overflow, correct? And with the fixed stored procedure, you do not see the errors? If you continue to have these severe problems on your production server even with the fixed stored procedures, then you should escalate your problem through PSS and that is the only channel that can issue a hotfix.
Steven
|||Actually the stored procedure is now fixed, i.e. it doesn't cause a stack overflow anymore. However, CLR stored procedures seem still crash the server. This has happened once. The error message in the stack dump of the SQL server logs says:"A fatal error occurred in .NET Framework runtime.". I'm not able to try the stored procedure on the server anymore, because it's our production server. I know that the stored procedure as such works OK, because it is run on many times a day on another server without problems.
All this should be said in the bug report a filed.|||
We installed a new server with SQL Server 2005 SP1 (let's call it X) and tried the CRL stored procedures there. The results was the same, the SQL Server immediatelly on X crashed when the stored procedure was run the first time. We updated X to SP2. After that we haven't been able to reproducude the crash on X. After a while we updated the original server to SP2. The problem hasn't occurred there either since the update.
We use ADOMDclient in the CLR stored procedures. The ADOMD DLL has been installed as an assembly in our SQL Server. When we updated the SQL Server to SP2, the ADOMDclient in GAC was updated and we had to update the same version to the assembly in SQL Server (the GAC must have the same version of the library in order the assembly to work inside of SQL Server). I wonder if this problem has something to do with ADOMDClient inside SQL Server....