Rendered at 21:08:42 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
GeertB 6 days ago [-]
Signed integer addition is only associative when overflow is defined to wrap around like unsigned arithmetic. This condition is matched here, because only debug builds panic on overflow. However, it's a bit of a gray area that the article completely ignores.
thrance 1 days ago [-]
Unlike C, Rust's signed arithmetic is fully specified to wrap around.
aw1621107 1 days ago [-]
> Rust's signed arithmetic is fully specified to wrap around.
Well, kind of. It's currently documented to wrap in release mode by default, but it's just that - a default. You're free to enable overflow checks in release mode (or disable them in debug if you really like oddball configurations), and either way overflow is considered a logic error that devs shouldn't rely on (and basically can't rely on when not in control of the end binary since it's the end user who controls overflow checks).
The Rust devs are theoretically open to making signed overflow panic by default, but consider such a change unlikely unless "something materially changes" [0].
Thanks for clarification, it seems Rust devs (as opposite to C devs) like good defaults and don't like making code accidentally cut yourself just because you looked at it wrong
pdpi 1 days ago [-]
This deserves elaborating on, because it's pretty cool.
Rust has two behaviours around overflow. In debug builds, it panics, in release builds it wraps.
IMO wrapping is a reasonable-enough behaviour to avoid UB in release builds, and panicking in debug is definitely the correct behaviour, because you're only avoiding UB by defaulting to something, but that's not nearly enough. In most applications where overflow is a risk you should make sure to choose what behaviour you consider correct.
Thankfully Rust has a pretty robust story around this:
fn add_behaviour() {
let small: i32 = 123;
let big: i32 = i32::MAX;
assert_eq!(small.wrapping_add(big), i32::MIN + 122);
assert_eq!(small.overflowing_add(big), (i32::MIN + 122, true));
assert_eq!(small.overflowing_add(small), (246, false));
assert_eq!(small.saturating_add(big), i32::MAX);
assert_panics!(a.strict_add(b)); // (nb: Not a real assertion)
}
And you could easily implement the default behaviour yourself with conditional compilation:
No. If you want wrapping, ask for it with Wrapping<T> or the specific Wrapping types, or the wrapping arithmetic APIs
It's true that since it's safe and faster, release builds default to wrapping rather than panic, but it's still wrong if you overflow any of Rust's default integer types, it's just that in a safe language it won't be Undefined Behaviour.
"I can't be bothered to do it correctly" speaks to the quality of the rest of the product, it's a Brown M&M [read about the Van Halen test if you don't know what a Brown M&M means]
pillmillipedes 1 days ago [-]
Unlike Rust and Cs before C23, C23's signed arithmetic is fully specified to wrap around.
edit - nevermind, I'm wrong lol
dzaima 1 days ago [-]
C23 specifies that signed integers must be two's complement, but still leaves signed arithmetic overflow as undefined behavior.
pillmillipedes 1 days ago [-]
yeah, turns out you're right. god dammit. maybe one day
wavemode 1 days ago [-]
Probably never. This has been debated to death by both C and C++ standards committees. The consensus is that, since signed overflow is almost always a sign of a bug in the program, keeping it undefined enables compilers to optimize by assuming it never happens, and also allows sanitizers to continue to flag it to developers so that they fix their bugs (though it could be argued that, a sanitizer doesn't really have to strictly adhere to the standard, the committee apparently didn't feel that way).
aw1621107 1 days ago [-]
> and also allows sanitizers to continue to flag it to developers so that they fix their bugs
I've never really found this argument particularly convincing; as you say, sanitizers don't have to strictly adhere to the standard, and they do in fact take advantage of this flexibility to check behaviors that "are not undefined behavior, but are often unintentional" (e.g., -fsanitize=unsigned-integer-overflow).
Makes me wonder whether "sanitizers can't flag defined behavior" is meant to be shorthand for some more nuanced position ("the false positive rate for signed overflow sanitizer checks would be too high", maybe?) or something else.
dzaima 1 days ago [-]
> -fsanitize=unsigned-integer-overflow
Of course, -fsanitize=unsigned-integer-overflow isn't enabled by default, and few people use it (github code search gives 6K results for that, compared to 175K for "-fsanitize=undefined"; which to be fair is a lot higher than I expected, but still not a lot).
> Makes me wonder whether "sanitizers can't flag defined behavior" is meant to be shorthand for some more nuanced position
And signed overflow checking would have to be off-by-default too, if people were allowed to start relying on it. It'd be less "false positive rate too high", more "it disallows you to use a genuine language feature that is actually useful", defeating the point of defining signed overflow in the first place.
(imo defining signed overflow specifically for reducing attack surface from exploitable UB is a mostly-separate discussion, which should not affect core language semantics, and certainly not what users would be suggested to do)
aw1621107 24 hours ago [-]
> Of course, -fsanitize=unsigned-integer-overflow isn't enabled by default, and few people use it
Sure, but it's still a counterexample for "you can't define it because it means sanitizers can't warn for it". Sanitizers can warn for it; you "just" get a worse signal-to-noise ratio.
> It'd be less "false positive rate too high", more "it disallows you to use a genuine language feature that is actually useful"
I'm not sure I see the distinction? Flagging a correct use of a language feature as incorrect is more or less the definition of a false positive, so if intentional signed overflows get a reasonable amount of use then that'd presumably result in an unacceptably noisy check to be enabled by default.
> defeating the point of defining signed overflow in the first place.
As for unsigned overflow checks I'd imagine the intent is that one would enable that particular check if you think that the corresponding overflow is more likely to be unintentional than not, and in the cases where it actually is intentional you can suppress the check.
dzaima 23 hours ago [-]
> it's still a counterexample for "you can't define it because it means sanitizers can't warn for it"
Sure, technically you can write a sanitizer for anything. It just becomes less a "sanitizer" you can always recommend everyone everywhere use, and more of just a heuristic thing that only really works if you design your code for its arbitrary desires.
> and in the cases where it actually is intentional you can suppress the check.
imo it'd be nice to have separate types for wrapping and non-wrapping integers for that, so that you have actual language-level semantics and an easy way to mix things (e.g. wrapping arith for hashing, mixed with non-wrapping arith for loop index or whatever) instead of suppressions.
aw1621107 11 hours ago [-]
[dead]
afdbcreid 22 hours ago [-]
C++ has introduced the concept of erroneous behavior for this, which is much better.
aw1621107 4 hours ago [-]
Making signed overflow EB would preclude using signed overflow for optimization though, which seems to be (correctly or not) considered an important use case.
rictic 1 days ago [-]
Yeah, it's my favorite bit of C lore, a program that reads two integers, adds them, and reports the sum isn't well defined.
1 days ago [-]
14113 1 days ago [-]
> Floating point math is often slower than integer math because the compiler is being conservative about how it optimizes your code.
It's not strictly true to say that it's "being conservative". What is more correct is to say that floating point operations have different semantics to integer operations, and an optimisation that retains the semantics of an expression over integers may not do so when applied to an expression over integers. Hence, it may be possible to apply one optimisation to an integer expression, but applying that to a floating-point expression may result in a different program meaning.
C/C++ compilers give you a way out of this with the `--ffast-math` flag, which essentially allows compilers to relax the constraints on floating-point optimisation passes.
> C/C++ compilers give you a way out of this with the -fast-math
People should really use the individual optimization flags they want (no signed zeros, no trapping math, associative math, reciprocal math) and not -ffast-math because the other optimizations it enables leads to surprising code (for example, isinf and isnan may become noops, which will break production code).
Basically never use an optimization flag that changes the semantics of your code without understanding exactly what that means. I have had to fix this in a number of codebases because someone thought that flag was as innocuous as -O3.
And if you have to enable flush to zero/denormals are zero it should be explicit in your code and scoped.
tialaramex 1 days ago [-]
Yes. In fact, even if you don't ask for semantics like the -fast-math flag, the floating point types are worth some extra time to understand before relying on them.
They're much stranger than the machine integers. The machine integers are basically like the Integers you were taught in school, except for overflow. That's not nothing but it's a complexity you can ignore entirely so long as you never overflow. In Rust you can have the language keep you safe - if an overflow occurs we'll panic and we're done. However the floating point types are a weird thing entirely invented for the convenience of the machine. They're too often introduced as if, like the machine integers, they're almost familiar numbers from school. Some languages even call these types "real" - but they very much are not actually the Real numbers, not even the approximation that the machine integers were to actual Integers. The programming language can't help you cope today. You can use software like "Herbie" to help you a bit, but today's languages just leave you with it.
Here's an easy example you saw in school, a tenth, written 0.1 in decimal. The floating point types cannot represent this number. When you ask for the 32-bit floating point value 0.1 in a language like C or Rust, you actually get exactly 0.100000001490116119384765625 because that was a number the type can represent and it was deemed "close enough".
afdbcreid 22 hours ago [-]
They're not real, they're rationals (well rationals are real but you get it).
If you want "simple" rationals, you can use the numerator/denominator scheme. This has its own problems but if you avoid overflows, they are the rationals you were taught in school, just like the integers.
The problem is that people (and languages) default to floating-point without understanding the consequences. Many times it does not matter and then floating point are indeed better (if you know how to use them, e.g. not comparing for equality), but sometimes it does.
jcranmer 16 hours ago [-]
> The problem is that people (and languages) default to floating-point without understanding the consequences.
If you were to insist on there being only one numeric data type in a language, then floating-point turns out to be the best compromise, especially because someone who doesn't understand the pitfalls of floating-point are going to be less likely to have it blow up in their face than other options. Fixed-point has a problem when the numbers have very different scales. Rational numbers don't let you do basic things like "measure the distance between two points" (because functions like sqrt or exp aren't defined on rational numbers).
tialaramex 13 hours ago [-]
> Rational numbers don't let you do basic things like "measure the distance between two points" (because functions like sqrt or exp aren't defined on rational numbers).
However the floating point types are just binary rationals, so if we took this "can't do basic things" at face value we couldn't do these operations on the floating point types either.
The reason they're so weird is a convenience to the implementation. I am not an EE so I can't tell you how much that saved, but it was a choice, obviously we can't implement the Reals because Almost All Reals aren't even Computable, but I think most software engineers really don't have an appropriate understanding of the floating point types and the result is buggy software.
jcranmer 8 hours ago [-]
The main selling point that people tout for using rational numbers over floating-point is their exactness, and the main thing people complain about floating-point is their approximate nature. Introduce the ability to do sqrt on rational numbers, and you somehow have to invent approximation for rational numbers. Choose the nearest representable rational number, and oops, nearly every operation on that rational number could overflow the numerator or denominator, so change that and oops now you've basically reinvented floating-point but worse.
Which is sort of the thing of floating-point: they may be weird, but it kind of turns out that they're ultimately less weird than all of their alternatives, if you try to do anything other than toy examples on them. It's almost like floating-point was designed by people who needed to do a lot of numerical work in computers instead of students in Programming 101!
tialaramex 7 hours ago [-]
As somebody who has implemented Hans Boehm's "Towards an API for the Real Numbers" paper I cannot agree that the floating point types are "less weird" than alternatives. They're much faster of course because they're implemented in hardware, but that's hardly surprising.
In my (unpublished) toy calculator I added a mechanism to bring an f32 or f64 into existence and convert it to my Rational type and so that's why I had the exact value of the f32 we call 0.1 to hand earlier. But of course the API goes much further than that. Want to multiply the square root of 40 by the square root of 1000? No problem that's exactly 20. A bright teenager can figure that out, and so can a properly designed API for numbers. The ordinary human activities you talked about of course work just fine in this API, it has historically been how the calculator in an Android phone works for example.
Symbolic approaches also have value in this space.
In both cases, what you do not get is the problem where the programmer mistakenly believes these types will behave like the numbers in school but actually they behave in an entirely different way. You have to go really far off piste, by which point likely the programmer has appreciated that they do not understand what's going on, before you trip on a hazard we cannot manage with this approach - whereas in traditional floating point that happens without them realising.
duped 1 days ago [-]
I mean I get that novice programmers might get tripped up on floating point representation but if you don't know "f32 can't represent 0.1 exactly" then you shouldn't yet be worried about the nuance of relaxing IEEE 754 compliance for the purposes of performance.
24 hours ago [-]
14113 1 days ago [-]
Yes, I was being a bit concise: Individual optimisations should be turned on as determined by profiling, application semantics, etc. My point was more that if you want to get as close as possible between integer and floating-point, then there is a flag that does it. That doesn't mean that you should do it, however...
rob74 1 days ago [-]
I'm not an expert on floating point math, but the "Does adding a small number do nothing?" example caught my eye, because the numbers used are constants, which are arbitrary precision in some languages (https://stackoverflow.com/questions/57511935/what-is-the-pur...). For instance, Go answers the question with false (but still prints out "1e+16" when you try to print 1e16 + 1): https://go.dev/play/p/mSAktWpCRJA
tialaramex 1 days ago [-]
Rust explicitly requires that constants are typed.
const FOO: f32 = 0.75; // The 32-bit floating point value three quarters
If you try
const UNTYPED = 0.75; // Does not compile, pick a type
I don't find the SO answer very convincing because it seems like it's trying to argue this is the Reals, and it just isn't, it's only a subset of the Rationals which happened to be convenient for Go to work with it. The Reals are much stranger.
rob74 1 days ago [-]
Real numbers include rational numbers (numbers which can be represented as a fraction of two integers) and irrational numbers (numbers that can't be represented as a fraction - the most famous one is probably π). Since irrational numbers have an infinite number of decimal places, they obviously can't be stored as a floating point value and also can't be written down exactly, no matter how many decimal places you use. "Arbitrary precision" constants might get you closer, but yes, you will never be able to store a "true" irrational number in a computer.
mswphd 21 hours ago [-]
it's worth mentioning the infinite number of decimal places isn't an issue. there is the formalism of computable numbers to get around this
roughly represent each number as a turing machine, which on input i outputs the ith digit. it works fine (it's slower than floats, but that's a different concern).
the issue is that the computable numbers are relatively small. in particular, there are countably many turing machines, so they're a countable subset (in fact subfield) of the reals. so in a precise sense they only make up a vanishingly small fraction of the real numbers. but they still capture many important mathematical constants, e.g. e and pi.
kibwen 1 days ago [-]
> you will never be able to store a "true" irrational number in a computer.
But yes, you can absolutely represent irrational numbers (only a finite amount of them of course). You can even do it symbolically.
ameliaquining 1 days ago [-]
Sorry, what exactly is your disagreement with the SO answer? It doesn't mention the reals.
pjmlp 1 days ago [-]
Ideally CPython would have a JIT that would be able to do this, depending on the current hardware like other ecosystems, but we are still not there yet.
IshKebab 1 days ago [-]
CPython can't do this because it's a change in semantics. You need explicit opt-in from the programmer.
Anyway adding this optimisation to CPython would be like putting active aero on a dandy horse.
pjmlp 1 days ago [-]
Some of us would like to have Python finally catch up to Lisp in compiler tooling, but alas.
As for change in semantics, apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures.
aw1621107 1 days ago [-]
> apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures.
I think it'd be an issue irrespective of the architecture? Optimizations are generally expected to preserve semantics and those languages all specify IEEE 754 semantics which aren't necessarily associative. For instance, from the Java language spec [0]:
> Floating-point arithmetic is carried out in accordance with the rules of the IEEE 754 Standard, including for overflow and underflow (§15.4), with the exception of the remainder operator % (§15.17.3).
Or the .NET reference [1]:
> The Double type complies with the IEC 60559:1989 (IEEE 754) standard for binary floating-point arithmetic.
Or the ECMAScript 2027 spec [2]:
> Numeric operators such as +, ×, =, and ≥ refer to those operations as determined by the type of the operands. [] When applied to Numbers, the operators refer to the relevant operations within IEEE 754-2019.
> Native Image now targets x86-64-v3 architecture by default on AMD64 and provides a new -march option to specify target compatibility. Use -march=compatibility for best compatibility or -march=native for best performance if a native executable is deployed on the same machine or on a machine with the same CPU features. To list all available machine types, use -march=list.
> Regardless, those optimisations are available on the respective JITs.
I'm pretty sure this is wrong. No JS engine, RyuJIT or or HotSpot break IEE-754. Java does have intrinsics for algebraic floating point operations (but does not apply them by default and I don't think they're exposed), the other I don't think.
jcranmer 1 days ago [-]
I did a deep dive a while back into the floating-point semantics of many languages, included JIT'd languages, which mostly uncovers that very few languages are particularly precise in their specification.
The tl;dr for the relevant languages here is:
* Java requires full IEEE 754 conformance (and bounds ULPs on java.lang.Math functions, although not fully correctly-rounded), although (now removed) strictfp permitted a slightly more relaxed mode to make it easier to implement using x87 FPU arithmetic.
* C# has license for excess precision mode and denormal flushing.
* JS is strict IEEE 754 conformance, except for math library functions (which can be more approximate).
* Go permits FMA contraction (but is otherwise silent).
* Most other interpreted/JIT'd languages pretty much go "you get your machine floating-point."
And, FWIW, all of those floating-point semantics are orthogonal to things enabled by -ffast-math or similar flags! The only languages that really discuss such things are C (in a TS nobody implements), Fortran (which lets you rearrange expressions as long as you preserve parentheses), Julia (which has a fast_fma-like function and a fast-math macro), and now Rust.
pklausler 21 hours ago [-]
Sadly, Fortran’s guarantee of the “integrity of parentheses” has been undermined lately by its standard committee.
aw1621107 4 hours ago [-]
What recent change(s) were made?
memming 1 days ago [-]
@fastmath in Julia!
pjmlp 1 days ago [-]
I linked the documentation....
afdbcreid 1 days ago [-]
I'm not sure what you tried to prove by those docs, but they don't say that they deviate from IEEE-754.
aw1621107 1 days ago [-]
> those optimisations are available on the respective JITs.
I'd assume they aren't applied by default and/or without the programmer explicitly opting in to those altered semantics, though. Would you be able to show otherwise?
I don't see how changing targeted instruction sets is relevant here as the instructions you use is orthogonal to whether you assume floating point operations are associative.
pjmlp 1 days ago [-]
I pasted the links for a reason.
aw1621107 1 days ago [-]
Neither of those appear to say anything about treating floating point operations as associative nor reordering them as a standard optimization, let alone allowing the programmer to opt into such an optimization.
I like the idea, but I hate how verbose it is. Would be nice if there was also a macro that would transform all arithmetic within a block into algebraic arithmetic. The name is also a bit misleading, as I would expect "alebraic_add" to give an exact algebraic result, but instead it enables optimisations based on associative semantics.
As a sidenote, if implemented as a macro, e.g.
fn fast_sum_f64(values: &[f64]) -> f64 {
let mut total = 0;
fp_opt!(associative, {
for value in values {
total += value;
}
});
return total;
}
A question arises what happens when operating on custom types that overload arithmetic operations. I think the cleanest approach here would be to let the custom type define optimised versions, or have another macro that automatically generates them based on the existing ones, i.e. propagate the optimisation flags.
afdbcreid 22 hours ago [-]
You can create an `struct Algebraic<T>(pub T)` newtype that overloads operators using algebraic methods. In fact such wrapper might be added to std (it was discussed but decided that not yet. Such wrappers exist for wrapping and saturating arithmetic).
Well, kind of. It's currently documented to wrap in release mode by default, but it's just that - a default. You're free to enable overflow checks in release mode (or disable them in debug if you really like oddball configurations), and either way overflow is considered a logic error that devs shouldn't rely on (and basically can't rely on when not in control of the end binary since it's the end user who controls overflow checks).
The Rust devs are theoretically open to making signed overflow panic by default, but consider such a change unlikely unless "something materially changes" [0].
[0]: https://github.com/rust-lang/rust/issues/47739#issuecomment-...
Rust has two behaviours around overflow. In debug builds, it panics, in release builds it wraps.
IMO wrapping is a reasonable-enough behaviour to avoid UB in release builds, and panicking in debug is definitely the correct behaviour, because you're only avoiding UB by defaulting to something, but that's not nearly enough. In most applications where overflow is a risk you should make sure to choose what behaviour you consider correct.
Thankfully Rust has a pretty robust story around this:
And you could easily implement the default behaviour yourself with conditional compilation:It's true that since it's safe and faster, release builds default to wrapping rather than panic, but it's still wrong if you overflow any of Rust's default integer types, it's just that in a safe language it won't be Undefined Behaviour.
"I can't be bothered to do it correctly" speaks to the quality of the rest of the product, it's a Brown M&M [read about the Van Halen test if you don't know what a Brown M&M means]
edit - nevermind, I'm wrong lol
I've never really found this argument particularly convincing; as you say, sanitizers don't have to strictly adhere to the standard, and they do in fact take advantage of this flexibility to check behaviors that "are not undefined behavior, but are often unintentional" (e.g., -fsanitize=unsigned-integer-overflow).
Makes me wonder whether "sanitizers can't flag defined behavior" is meant to be shorthand for some more nuanced position ("the false positive rate for signed overflow sanitizer checks would be too high", maybe?) or something else.
Of course, -fsanitize=unsigned-integer-overflow isn't enabled by default, and few people use it (github code search gives 6K results for that, compared to 175K for "-fsanitize=undefined"; which to be fair is a lot higher than I expected, but still not a lot).
> Makes me wonder whether "sanitizers can't flag defined behavior" is meant to be shorthand for some more nuanced position
And signed overflow checking would have to be off-by-default too, if people were allowed to start relying on it. It'd be less "false positive rate too high", more "it disallows you to use a genuine language feature that is actually useful", defeating the point of defining signed overflow in the first place.
(imo defining signed overflow specifically for reducing attack surface from exploitable UB is a mostly-separate discussion, which should not affect core language semantics, and certainly not what users would be suggested to do)
Sure, but it's still a counterexample for "you can't define it because it means sanitizers can't warn for it". Sanitizers can warn for it; you "just" get a worse signal-to-noise ratio.
> It'd be less "false positive rate too high", more "it disallows you to use a genuine language feature that is actually useful"
I'm not sure I see the distinction? Flagging a correct use of a language feature as incorrect is more or less the definition of a false positive, so if intentional signed overflows get a reasonable amount of use then that'd presumably result in an unacceptably noisy check to be enabled by default.
> defeating the point of defining signed overflow in the first place.
As for unsigned overflow checks I'd imagine the intent is that one would enable that particular check if you think that the corresponding overflow is more likely to be unintentional than not, and in the cases where it actually is intentional you can suppress the check.
Sure, technically you can write a sanitizer for anything. It just becomes less a "sanitizer" you can always recommend everyone everywhere use, and more of just a heuristic thing that only really works if you design your code for its arbitrary desires.
> and in the cases where it actually is intentional you can suppress the check.
imo it'd be nice to have separate types for wrapping and non-wrapping integers for that, so that you have actual language-level semantics and an easy way to mix things (e.g. wrapping arith for hashing, mixed with non-wrapping arith for loop index or whatever) instead of suppressions.
It's not strictly true to say that it's "being conservative". What is more correct is to say that floating point operations have different semantics to integer operations, and an optimisation that retains the semantics of an expression over integers may not do so when applied to an expression over integers. Hence, it may be possible to apply one optimisation to an integer expression, but applying that to a floating-point expression may result in a different program meaning.
C/C++ compilers give you a way out of this with the `--ffast-math` flag, which essentially allows compilers to relax the constraints on floating-point optimisation passes.
For an example of how this works in GCC, take a look here: https://gcc.gnu.org/wiki/FloatingPointMath
People should really use the individual optimization flags they want (no signed zeros, no trapping math, associative math, reciprocal math) and not -ffast-math because the other optimizations it enables leads to surprising code (for example, isinf and isnan may become noops, which will break production code).
Basically never use an optimization flag that changes the semantics of your code without understanding exactly what that means. I have had to fix this in a number of codebases because someone thought that flag was as innocuous as -O3.
And if you have to enable flush to zero/denormals are zero it should be explicit in your code and scoped.
They're much stranger than the machine integers. The machine integers are basically like the Integers you were taught in school, except for overflow. That's not nothing but it's a complexity you can ignore entirely so long as you never overflow. In Rust you can have the language keep you safe - if an overflow occurs we'll panic and we're done. However the floating point types are a weird thing entirely invented for the convenience of the machine. They're too often introduced as if, like the machine integers, they're almost familiar numbers from school. Some languages even call these types "real" - but they very much are not actually the Real numbers, not even the approximation that the machine integers were to actual Integers. The programming language can't help you cope today. You can use software like "Herbie" to help you a bit, but today's languages just leave you with it.
https://herbie.uwplse.org/
Here's an easy example you saw in school, a tenth, written 0.1 in decimal. The floating point types cannot represent this number. When you ask for the 32-bit floating point value 0.1 in a language like C or Rust, you actually get exactly 0.100000001490116119384765625 because that was a number the type can represent and it was deemed "close enough".
If you want "simple" rationals, you can use the numerator/denominator scheme. This has its own problems but if you avoid overflows, they are the rationals you were taught in school, just like the integers.
The problem is that people (and languages) default to floating-point without understanding the consequences. Many times it does not matter and then floating point are indeed better (if you know how to use them, e.g. not comparing for equality), but sometimes it does.
If you were to insist on there being only one numeric data type in a language, then floating-point turns out to be the best compromise, especially because someone who doesn't understand the pitfalls of floating-point are going to be less likely to have it blow up in their face than other options. Fixed-point has a problem when the numbers have very different scales. Rational numbers don't let you do basic things like "measure the distance between two points" (because functions like sqrt or exp aren't defined on rational numbers).
However the floating point types are just binary rationals, so if we took this "can't do basic things" at face value we couldn't do these operations on the floating point types either.
The reason they're so weird is a convenience to the implementation. I am not an EE so I can't tell you how much that saved, but it was a choice, obviously we can't implement the Reals because Almost All Reals aren't even Computable, but I think most software engineers really don't have an appropriate understanding of the floating point types and the result is buggy software.
Which is sort of the thing of floating-point: they may be weird, but it kind of turns out that they're ultimately less weird than all of their alternatives, if you try to do anything other than toy examples on them. It's almost like floating-point was designed by people who needed to do a lot of numerical work in computers instead of students in Programming 101!
In my (unpublished) toy calculator I added a mechanism to bring an f32 or f64 into existence and convert it to my Rational type and so that's why I had the exact value of the f32 we call 0.1 to hand earlier. But of course the API goes much further than that. Want to multiply the square root of 40 by the square root of 1000? No problem that's exactly 20. A bright teenager can figure that out, and so can a properly designed API for numbers. The ordinary human activities you talked about of course work just fine in this API, it has historically been how the calculator in an Android phone works for example.
Symbolic approaches also have value in this space.
In both cases, what you do not get is the problem where the programmer mistakenly believes these types will behave like the numbers in school but actually they behave in an entirely different way. You have to go really far off piste, by which point likely the programmer has appreciated that they do not understand what's going on, before you trip on a hazard we cannot manage with this approach - whereas in traditional floating point that happens without them realising.
https://en.wikipedia.org/wiki/Computable_number
roughly represent each number as a turing machine, which on input i outputs the ith digit. it works fine (it's slower than floats, but that's a different concern).
the issue is that the computable numbers are relatively small. in particular, there are countably many turing machines, so they're a countable subset (in fact subfield) of the reals. so in a precise sense they only make up a vanishingly small fraction of the real numbers. but they still capture many important mathematical constants, e.g. e and pi.
Joke's on you, in my programming language all numbers are written in phinary: https://en.wikipedia.org/wiki/Golden_ratio_base
But yes, you can absolutely represent irrational numbers (only a finite amount of them of course). You can even do it symbolically.
Anyway adding this optimisation to CPython would be like putting active aero on a dandy horse.
As for change in semantics, apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures.
I think it'd be an issue irrespective of the architecture? Optimizations are generally expected to preserve semantics and those languages all specify IEEE 754 semantics which aren't necessarily associative. For instance, from the Java language spec [0]:
> Floating-point arithmetic is carried out in accordance with the rules of the IEEE 754 Standard, including for overflow and underflow (§15.4), with the exception of the remainder operator % (§15.17.3).
Or the .NET reference [1]:
> The Double type complies with the IEC 60559:1989 (IEEE 754) standard for binary floating-point arithmetic.
Or the ECMAScript 2027 spec [2]:
> Numeric operators such as +, ×, =, and ≥ refer to those operations as determined by the type of the operands. [] When applied to Numbers, the operators refer to the relevant operations within IEEE 754-2019.
[0]: https://docs.oracle.com/javase/specs/jls/se26/jls26.pdf
[1]: https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
[2]: https://tc39.es/ecma262/#sec-mathematical-operations
RyuJIT target CPU modes, breaking change due to dropping support for older hardware,
https://github.com/dotnet/docs/issues/48045
> Native Image now targets x86-64-v3 architecture by default on AMD64 and provides a new -march option to specify target compatibility. Use -march=compatibility for best compatibility or -march=native for best performance if a native executable is deployed on the same machine or on a machine with the same CPU features. To list all available machine types, use -march=list.
https://www.graalvm.org/release-notes/JDK_20
I'm pretty sure this is wrong. No JS engine, RyuJIT or or HotSpot break IEE-754. Java does have intrinsics for algebraic floating point operations (but does not apply them by default and I don't think they're exposed), the other I don't think.
The tl;dr for the relevant languages here is:
* Java requires full IEEE 754 conformance (and bounds ULPs on java.lang.Math functions, although not fully correctly-rounded), although (now removed) strictfp permitted a slightly more relaxed mode to make it easier to implement using x87 FPU arithmetic.
* C# has license for excess precision mode and denormal flushing.
* JS is strict IEEE 754 conformance, except for math library functions (which can be more approximate).
* Go permits FMA contraction (but is otherwise silent).
* Most other interpreted/JIT'd languages pretty much go "you get your machine floating-point."
And, FWIW, all of those floating-point semantics are orthogonal to things enabled by -ffast-math or similar flags! The only languages that really discuss such things are C (in a TS nobody implements), Fortran (which lets you rearrange expressions as long as you preserve parentheses), Julia (which has a fast_fma-like function and a fast-math macro), and now Rust.
I'd assume they aren't applied by default and/or without the programmer explicitly opting in to those altered semantics, though. Would you be able to show otherwise?
I don't see how changing targeted instruction sets is relevant here as the instructions you use is orthogonal to whether you assume floating point operations are associative.
As a sidenote, if implemented as a macro, e.g.
A question arises what happens when operating on custom types that overload arithmetic operations. I think the cleanest approach here would be to let the custom type define optimised versions, or have another macro that automatically generates them based on the existing ones, i.e. propagate the optimisation flags.