 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
.NET Forum / .NET Framework / New Users / January 2008Double is doing incorrect subtraction
I have the following code: Dim x As Double = 45.333 Dim y As Double = 45 Dim z As Double = x - y Me.Label1.Text = z.ToString() The result should obviously be 0.333, but the value displayed in Me.Label1.Text is: 0.332999999999998 I have seen stuff like this in other situations, but I never managed to figure out how to fix it. Can somebody help me here?  SignatureNathan Sokalski njsokalski@hotmail.com http://www.nathansokalski.com/ >I have the following code: > [quoted text clipped - 10 lines] > I have seen stuff like this in other situations, but I never managed to > figure out how to fix it. Can somebody help me here? In order to show the correct result you have to format it. For example you could do this: Me.Label1.Text =z.ToString("0.000") Doubles are floating point numbers and that causes this behavior. If you want to find more information try keyword "IEEE 754" with Google. -Teemu Consider using decimal instead of double if you want to get the answer you expect below. > I have the following code: > [quoted text clipped - 10 lines] > I have seen stuff like this in other situations, but I never managed to > figure out how to fix it. Can somebody help me here? > I have the following code: > [quoted text clipped - 10 lines] > I have seen stuff like this in other situations, but I never managed to > figure out how to fix it. Can somebody help me here? Double is doing entirely correct subtraction. The problem is that Double is an implementation of IEEE floating point, and IEEE floating point is incapable of representing all numbers exactly. Read this: http://support.microsoft.com/kb/42980 Nathan, This is basic Float, Double or whatever floating poing calculation for mathimatical programming, Decimal for business programming, Cor "Nathan Sokalski" <njsokalski@hotmail.com> schrieb: > Dim x As Double = 45.333 > Dim y As Double = 45 [quoted text clipped - 8 lines] > I have seen stuff like this in other situations, but I never managed to > figure out how to fix it. Can somebody help me here? In addition to the other replies, take a look at these articles: IEEE Standard 754 Floating Point Numbers <URL:http://steve.hollasch.net/cgindex/coding/ieeefloat.html> SignatureM S Herfried K. Wagner M V P <URL:http://dotnet.mvps.org/> V B <URL:http://dotnet.mvps.org/dotnet/faqs/>
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.