+ Reply to Thread
Results 1 to 5 of 5

incorrect calculation

Hybrid View

  1. #1
    Robin
    Guest

    incorrect calculation

    I am experiencing a bizarre phenomenon in Excel. I received a file from a
    customer and there is an instance where, multiplying two cells together, the
    answer is incorrect by about 0.02%. I am dealing with large numbers so I
    cannot ignore this. I am not a beginner with Excel and have checked all
    possibilities such as ensuring cell format is number and not text, etc. Its
    almost like its rounding the values off in the background, but its not. Has
    anyone got any ideas?

  2. #2
    bj
    Guest

    RE: incorrect calculation

    is it set for precision as displayed?
    what are the numbers that are being multiplied together and what are the
    results?

    "Robin" wrote:

    > I am experiencing a bizarre phenomenon in Excel. I received a file from a
    > customer and there is an instance where, multiplying two cells together, the
    > answer is incorrect by about 0.02%. I am dealing with large numbers so I
    > cannot ignore this. I am not a beginner with Excel and have checked all
    > possibilities such as ensuring cell format is number and not text, etc. Its
    > almost like its rounding the values off in the background, but its not. Has
    > anyone got any ideas?


  3. #3
    Ron Rosenfeld
    Guest

    Re: incorrect calculation

    On Wed, 21 Sep 2005 04:14:01 -0700, "Robin" <Robin@discussions.microsoft.com>
    wrote:

    >I am experiencing a bizarre phenomenon in Excel. I received a file from a
    >customer and there is an instance where, multiplying two cells together, the
    >answer is incorrect by about 0.02%. I am dealing with large numbers so I
    >cannot ignore this. I am not a beginner with Excel and have checked all
    >possibilities such as ensuring cell format is number and not text, etc. Its
    >almost like its rounding the values off in the background, but its not. Has
    >anyone got any ideas?


    How large are the numbers?

    Are you running into Excel precision limitations?

    What numbers are you multiplying and what are the results?

    When you look at the accuracy of the calculation, you need to look at the
    numbers with regard to the precision Excel is using for the process (15 decimal
    places) and not the displayed number.


    --ron

  4. #4
    Jerry W. Lewis
    Guest

    Re: incorrect calculation

    I am not aware of any instances where Excel's basic arithmetic does not
    work correctly. Please post your equation.

    Jerry

    Robin wrote:

    > I am experiencing a bizarre phenomenon in Excel. I received a file from a
    > customer and there is an instance where, multiplying two cells together, the
    > answer is incorrect by about 0.02%. I am dealing with large numbers so I
    > cannot ignore this. I am not a beginner with Excel and have checked all
    > possibilities such as ensuring cell format is number and not text, etc. Its
    > almost like its rounding the values off in the background, but its not. Has
    > anyone got any ideas?



  5. #5
    Robin
    Guest

    Re: incorrect calculation

    It turns out the "precision as displayed" box was ticked. This has corrected
    the problem. Thanks for the useful tips.

    "Jerry W. Lewis" wrote:

    > I am not aware of any instances where Excel's basic arithmetic does not
    > work correctly. Please post your equation.
    >
    > Jerry
    >
    > Robin wrote:
    >
    > > I am experiencing a bizarre phenomenon in Excel. I received a file from a
    > > customer and there is an instance where, multiplying two cells together, the
    > > answer is incorrect by about 0.02%. I am dealing with large numbers so I
    > > cannot ignore this. I am not a beginner with Excel and have checked all
    > > possibilities such as ensuring cell format is number and not text, etc. Its
    > > almost like its rounding the values off in the background, but its not. Has
    > > anyone got any ideas?

    >
    >


+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1