• sgh
    link
    fedilink
    English
    arrow-up
    7
    arrow-down
    7
    ·
    edit-2
    6 months ago

    I don’t see any reply from OP so I’m growing confident that what you’re talking about is not OP’s point.

    Often times when coding you may want to quickly write down 2MB but you may need to type it in bytes, so either you calculate 2 * 1024 * 1024 while coding, or you remember the number 2097152.

    Now, since 2097152 is not such a common number that one would remember, you may quickly turn to the globally acclaimed oracle search engine to get such an answer, but all you get is a number in scientific notation, approximated, without an option to read it in standard decimal base. So you have to open the calculator and ask the same question again to get the answer you need.

    If it helps, try to ignore what’s in the search bar and tell me if it makes more sense.

    Edit: Additionally, if you were to NOT use the scientific notation, the length of the result would be shorter:

    2,097e+6 (8 characters) vs 2097152 (7 characters)

    • jas0n@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      6 months ago

      Yeah. This was the problem. i just wanted to copy and paste it quickly but they rounded it off. It’s a useless conversion. And I switched it to mebibytes as well since that’s what everyone really means when they say megabytes unless you’re making selling storage devices. =]

      Indeed, I should have just used my calculator program.

    • thesmokingman@programming.dev
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      3
      ·
      6 months ago

      If I ignore what’s in the search bar, I remember that the prefix “mebi” means 2^20 and use a calculator. Your point doesn’t make sense because you’re asking us to get mad at a tool intended to convert scientific units for using the bog standard scientific notation. Byte math uses powers of 2 ergo we should use a calculator that isn’t explicitly set up for rounding.

      • sgh
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        6
        ·
        6 months ago

        Are you telling me that whenever you work with Digital Storage units you should never use scientific notation?

        • thesmokingman@programming.dev
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          3
          ·
          6 months ago

          You literally spent an entire comment explaining why you should not use scientific notation and now you’re asking why I might prefer precision in byte arithmetic?

          Good luck with that.

          • sgh
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            6
            ·
            edit-2
            6 months ago

            Nope, au contraire, I agree, I’m just pointing out that you said that digital storage conversion should happen in non-scientific notation, so you should now agree with OP in that Google is choosing the wrong output format for a, quote from the screenshot, “Digital Storage” conversion.

            And yes, I’m writing multiple comments trying to explain this through narrative, without having to point out what in your reasoning sounds stupid.

            I.E. Now don’t you tell me that Google is incapable of figuring out which output format it should use for such a calculation…

            Since I apparently need to explain this like you’re 5, please read my last comment like the following:

            “Are you now agreeing with me/OP that whenever you work with Digital Storage units you should never use scientific notation?”