• xigoi@lemmy.sdf.org
    link
    fedilink
    arrow-up
    2
    ·
    10 months ago

    If you’re working with floating point, you should be aware it’s just an approximation.

    • kevincox@lemmy.ml
      link
      fedilink
      arrow-up
      1
      ·
      10 months ago

      People think floats are too magical. Calling it an approximation is sort of leaning into this. Floats have limited precision like every other fixed size representation of a number.

      This is sort of saying that integers are an approximation because int(1.6) + int(2.6) = 5. What do you mean‽ Clearly 1.6 + 2.6 = 4.2 ~= 4!

      Floating points can’t perfectly represent 0.1 or 0.2 much like integers can’t represent 1.6. So it is rounded to the nearest representable value. The addition is then performed perfectly accurately using these values and the result is the rounded to the nearest representable value. (Much like integers division is “rounded”). That result happens to not be equal to the nearest representable value to 0.3.

      It is definitely a bit surprising. And yes, because of rounding it is technically approximate. But the way people talk about floating point makes it sound like it is some nebulous thing that gives a different result every time or just sort of does the correct thing.

      • r00ty@kbin.life
        link
        fedilink
        arrow-up
        1
        ·
        10 months ago

        I think specifically, they have amazing precision. But the boundaries just don’t fall perfectly on round numbers we humans would expect. That’s what gets people confused.

        Rounding can resolve these problems, or don’t use float if you don’t need to.

  • psud@aussie.zone
    link
    fedilink
    arrow-up
    0
    ·
    10 months ago

    If working in currency, work in cents and divide by 100 and round to 2 decimals for output