SqlDecimal calculations in .NET 2.0

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Beat Bucheli - 13 Sep 2006 15:41 GMT

Hello,

I'm trying to do some simple calculations with System.Data.SqlDecimal on
.NET 2.0:

SqlDecimal one = new SqlDecimal(1);
SqlDecimal seven = new SqlDecimal(7);
SqlDecimal oneBySeven = SqlDecimal.Divide(one, seven);

The precision of the result in oneBySeven is way below my expectations. I
get 0.142857, but I expected something like 0.1428571428571428571428571429
or even more precise. Additional values reported by the debugger:

// Data[0] Data[1] Data[2] Data[3] Precision
Scale Value
// one 0x00000001 0x00000000 0x00000000 0x00000000 1 0
{1}
// seven 0x00000007 0x00000000 0x00000000 0x00000000 1 0
{7}
// oneBySeven 0x00022e09 0x00000000 0x00000000 0x00000000 7 6
{0.142857}

Any explanations? Thanks!
Beat

Reply to this Message

David Browne - 13 Sep 2006 16:38 GMT

> I'm trying to do some simple calculations with System.Data.SqlDecimal on
> .NET 2.0:
[quoted text clipped - 18 lines]
> Any explanations? Thanks!
> Beat

Sure. This explains all:

Precision, Scale, and Length (Transact-SQL)
http://msdn2.microsoft.com/en-us/library/ms190476.aspx

Given decimals e1 and e1, with precision p1 and p2 and scale s1 and s2
respectively, the quotient (e1 / e2)
will have precision = p1 - s1 + s2 + max(6, s1 + p2 + 1) and scale = max(6,
s1 + p2 + 1)

In a table or in TSQL a decimal type is declared with a precision and scale.
When you use this .NET constructor the precision and scale are inferred from
the constructor argument. If you want more precision in the quotient you
can specify more precision in the arguments:

SqlDecimal one = SqlDecimal.ConvertToPrecScale(new SqlDecimal(1), 20,
12);
SqlDecimal seven = SqlDecimal.ConvertToPrecScale(new SqlDecimal(7), 20,
12); ;
SqlDecimal oneBySeven = SqlDecimal.Divide(one, seven);

Console.WriteLine(oneBySeven);

David

Reply to this Message

SqlDecimal calculations in .NET 2.0

Free Magazines