Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / Languages / VB.NET / September 2005

Tip: Looking for answers? Try searching our database.

VB 6.3 Int Problem

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Chris Prior - 12 Sep 2005 03:23 GMT
Since the last update for office 2003, i've been having issues with the int()
function in vb (version 9972 vba retail 6.4.992)

The accounting software that the business i work for writes for dozens of
clients regularly uses x = (int(x*100))/100 to ensure that there are no
fractional cents on the end of prices. It works wonderfully, until one of a
number of values are used. 8.95, 9.95, 2094.95, and a host of other seemingly
random numbers. For these, the end value of x computes as 8.94, 9.94 etc. Is
this something other people have come across??
Cor Ligthert [MVP] - 12 Sep 2005 06:18 GMT
Chris,

Maybe I read your message wrong, however this is a dotNet VB language
newsgroup.

Probably do you need help on Excel, for which is a very active programmer
newsgroup.

microsoft.public.excel.programmer

If I am right wiht my assuption than I give you much more change on your
question than here.

Cor
Andrew Taylor - 12 Sep 2005 11:54 GMT
[Without quoting - Original post was about problem with
x = (int(x*100))/100, where 8.95 >>> 8.94 in Excel VBA]

> Chris,
>
[quoted text clipped - 10 lines]
>
> Cor

However, the same problem could occur in .NET:

In Visual Studio Command window:
? 8.95*100
894.99999999999989
? 9.95*100
994.99999999999989

As always with this type of problem, the answer is that you can't rely
on floating point numbers to give exact results.

Andrew Taylor
Herfried K. Wagner [MVP] - 12 Sep 2005 12:26 GMT
"Andrew Taylor" <andrew.taylor@cantab.net> schrieb:
> As always with this type of problem, the answer is that you can't rely
> on floating point numbers to give exact results.

Additional information:

<URL:http://groups.google.de/group/microsoft.public.dotnet.languages.vb/msg/1d103c859
0802792
>

Signature

M S   Herfried K. Wagner
M V P  <URL:http://dotnet.mvps.org/>
V B   <URL:http://classicvb.org/petition/>

AMDRIT - 12 Sep 2005 16:19 GMT
Here is some more explaination that doesn't require deep thought

Option Explicit

Sub SimpleTest()

Dim SomeNumberA As Double
Dim SomeNumberB As Currency
Dim SomeNumberC As Single

SomeNumberA = 8.950568
SomeNumberB = 8.950568
SomeNumberC = 8.950568

'Floating points are only usefull for scientific notation (Single and
Double)
Debug.Print "Single " & (Int(SomeNumberC * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberC * 100)) / 100)

Debug.Print "Double " & (Int(SomeNumberA * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberA * 100)) / 100)

Debug.Print "Rounded Double " & (Int(Round(SomeNumberA, 2) * 100)) / 100 & "
is a " & TypeName((Int(Round(SomeNumberA, 2) * 100)) / 100)

'Scaled values should be used when precision counts
' Visual basic does not have a decimal value type
' but it does have a CDec (Convert to Decimal) operator
Debug.Print "Decimal " & (Int(CDec(SomeNumberA) * 100)) / 100 & " is a " &
TypeName((Int(CDec(SomeNumberA) * 100)) / 100)

' Currency Value types are the next best thing and will maintain precision
' notice this becomes a double again after all the operations are done
Debug.Print "Currency " & (Int(SomeNumberB * 100)) / 100 & " is a " &
TypeName((Int(SomeNumberB * 100)) / 100)

' A quick work around to your delima
Debug.Print "Rounded Decimal " & Round(CDec(SomeNumberA), 2) & " is a " &
TypeName(Round(CDec(SomeNumberA), 2))

' Another way to skin the cat
Debug.Print "Special " & SpecialReduce(SomeNumberA, 2) & " is a " &
TypeName(SpecialReduce(SomeNumberA, 2))

End Sub

'
Public Function SpecialReduce(InputValue As Variant, Places As Integer) As
Currency

If Not IsNumeric(InputValue) Then
   Err.Raise 13, "My Module", "InputValue must be of numeric type."
End If

SpecialReduce = Round(CCur(InputValue), Places)

End Function

> "Andrew Taylor" <andrew.taylor@cantab.net> schrieb:
>> As always with this type of problem, the answer is that you can't rely
[quoted text clipped - 3 lines]
>
> <URL:http://groups.google.de/group/microsoft.public.dotnet.languages.vb/msg/1d103c859
0802792
>

Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.