+ Reply to Thread
Results 1 to 4 of 4

Excel 2007 : 300.00 - 288.60 = 11.40000000000003.5527136788005

  1. #1
    Registered User
    Join Date
    05-09-2012
    Location
    New Zealand
    MS-Off Ver
    Excel 2003 and Excel 2007
    Posts
    2

    300.00 - 288.60 = 11.40000000000003.5527136788005

    I get this problem with both 2003 and 2007 excel on 2 different computers.

    Type this into Excel:

    A1=300
    A2=288.6
    A3=A1-A2

    are you with me so far? The answer is 11.40 right, it even says that on the screen. But is it really?
    Keep typing:

    A4=11.40
    A5=A3-A4

    Wait, 11.40 - 11.40 = 2.30926E-14

    I think Excel needs to go back to school and learn math

  2. #2
    Forum Expert Paul's Avatar
    Join Date
    02-05-2007
    Location
    Wisconsin
    MS-Off Ver
    2016/365
    Posts
    6,887

    Re: 300.00 - 288.60 = 11.40000000000003.5527136788005

    Hi ddd, welcome to the forum.

    See this Microsoft article on why this happens in Excel (and other software that uses binary numbers for floating point calculations):

    http://support.microsoft.com/kb/78113

    It is "relatively" common knowledge as you go up the Excel ladder in life.

  3. #3
    Administrator FDibbins's Avatar
    Join Date
    12-29-2011
    Location
    Duncansville, PA USA
    MS-Off Ver
    Excel 7/10/13/16/365 (PC ver 2310)
    Posts
    53,048

    Re: 300.00 - 288.60 = 11.40000000000003.5527136788005

    interesting, when i do =EXACT(A4,A3) i get true, so the values are the same
    1. Use code tags for VBA. [code] Your Code [/code] (or use the # button)
    2. If your question is resolved, mark it SOLVED using the thread tools
    3. Click on the star if you think someone helped you

    Regards
    Ford

  4. #4
    Registered User
    Join Date
    05-09-2012
    Location
    New Zealand
    MS-Off Ver
    Excel 2003 and Excel 2007
    Posts
    2

    Re: 300.00 - 288.60 = 11.40000000000003.5527136788005

    Thanks for that :-)

    I solved the problem by using the "Precision as Displayed" option
    Excel 2003 > Options > Tools > Calculation > Precision as Displayed
    and
    Excel 2007 Options > Advanced > Set Precision as Displayed

    Cheers

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1