Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sneaky signed ± anti-overflow #118

Closed
oscbyspro opened this issue Jun 6, 2023 · 0 comments
Closed

Sneaky signed ± anti-overflow #118

oscbyspro opened this issue Jun 6, 2023 · 0 comments
Labels
bug please fix this :(
Milestone

Comments

@oscbyspro
Copy link
Owner

oscbyspro commented Jun 6, 2023

From oscbyspro/Numberick#3:

The current addition and subtraction methods have this structure:

@inlinable public mutating func addReportingOverflow(_ other: Self) -> Bool {
    let a: Bool = self.low .addReportingOverflow(other.low )
    let b: Bool = self.high.addReportingOverflow(other.high)
    let c: Bool = a && self.high.addReportingOverflow(1 as Digit)
    return b || c // this should be « b != c » (!)
}

The problem is that b and c will negate each other when Self is signed. See regression tests below:

NBKAssertAddition   (T(descending: HL(.max, .max)), T(-1), T(descending: HL(.max, .max - 1)), false) // carry 1st
NBKAssertAddition   (T(descending: HL(.min, .max)), T(-1), T(descending: HL(.min, .max - 1)), false) // carry 2nd

NBKAssertSubtraction(T(descending: HL(.min, .min)), T(-1), T(descending: HL(.min, .min + 1)), false) // carry 1st
NBKAssertSubtraction(T(descending: HL(.max, .min)), T(-1), T(descending: HL(.max, .min + 1)), false) // carry 2nd

These tests validate the case where c appears after b as well as the case where c appears before b.

@oscbyspro oscbyspro added the bug please fix this :( label Jun 6, 2023
@oscbyspro oscbyspro added this to the v3.0.0 milestone Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug please fix this :(
Projects
None yet
Development

No branches or pull requests

1 participant