+ Reply to Thread
Results 1 to 3 of 3

Losing significant digits when multiplying???

Hybrid View

  1. #1
    Registered User
    Join Date
    07-14-2010
    Location
    Snowmass, Colorado
    MS-Off Ver
    Excel 2007
    Posts
    30

    Losing significant digits when multiplying???

    Hi All,

    I have a question which hopefully has a simple answer. Why when I multiply two numbers together am I losing all significant digits after the decimal point. I'll show you my code, I'll show you what I want to happen and then I'll show you what is happening. Thanks so much for reading.

    Here is a small snippet of my code:

    Dim Derr As Long
    Dim aa As Integer
    Dim C As Long
    Dim result As Single 'I think this is causing the problem - but this must be a single, see below...
    Dim f As String
    Dim g As String
    Dim i As Integer
    Dim newresult As Long
    
    'Calling a function from an external dll where result MUST be a single for it to work properly
    Derr = D2R_VBGetSingleResult(a, b, C, result, 1, f, g)
    
    Debug.Print C
    Debug.Print result
    Debug.Print .Cells(row + 4, 5).Value
    newresult = result * .Cells(row + 4, 5).Value
    Debug.Print newresult
     .Cells(row + 4, aa + 4).Value = newresult
                  
      'apply formatting
     .Cells(row + 4, aa + 4).NumberFormat = "#.0"
    So, if you look at my code above, result is 5,020,861
    Cells(row + 4, 5).Value is 6.83947883572414E-06 (or 0.00000683947883572414)

    The result I would like to get is: 34.3 (34.3400........... formatted to have one decimal place).

    What I AM getting is 34.0 (newresult is 34).

    Is it because I am multiplying by a single and do I need to cast this to another data type before multiplying? Help!!

    Thanks!
    BRL

  2. #2
    Forum Expert shg's Avatar
    Join Date
    06-20-2007
    Location
    The Great State of Texas
    MS-Off Ver
    2010, 2019
    Posts
    40,689

    Re: Losing significant digits when multiplying???

    The variable newresult is declared as a Long, which is a whole number (a 32-bit signed integer).

    Cells with numeric values are Doubles.
    Entia non sunt multiplicanda sine necessitate

  3. #3
    Registered User
    Join Date
    07-14-2010
    Location
    Snowmass, Colorado
    MS-Off Ver
    Excel 2007
    Posts
    30

    Re: Losing significant digits when multiplying???

    That was it! You rock! Thank you!!!!

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1