Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The round() function has issues when trying to do places=1 #14

Closed
sandalle opened this issue Jul 16, 2014 · 0 comments
Closed

The round() function has issues when trying to do places=1 #14

sandalle opened this issue Jul 16, 2014 · 0 comments
Labels

Comments

@sandalle
Copy link
Owner

In the debug log for 0.3.9, trying to round 99.93125 to one place (we want 99.9):

In round(num=99.93125,places=1), i=4, num=99.9, newNum=, digit=9.
In round(num=99.93125,places=1), i=3, num=99.9, newNum=0, digit=0.9.
Called as round(num=99.93125,places=1), returns "991.90"

While other numbers appear to work:

In round(num=0.6069465,places=3), i=5, num=0.606, newNum=, digit=6.
Called as round(num=0.6069465,places=3), returns "0.607"
-- round() function from
-- http://www.computercraft.info/forums2/index.php?/topic/4023-lua-printformat-with-floating-point-numbers/page__view__findpost__p__31037
local function round(num, places)
    local origNum = num
    num = tostring(num)
    local inc = false

    -- Make sure decimal is a valid integer for later arithmetic
    local decimal = string.find(num, "%.") or 0

    if (num:len() - decimal) <= places then
        printLog("Called as round(num="..origNum..",places="..places.."), already rounded, returns \""..num.."\"")
        return tonumber(num)
    end --already rounded, nothing to do.

    local digit = tonumber(num:sub(decimal + places + 1))
    num = num:sub(1, decimal + places)

    if digit <= 4 then
        printLog("Called as round(num="..origNum..",places="..places.."), no incrementing needed, returns \""..origNum.."\"")
        return tonumber(origNum)
    end --no incrementation needed, return truncated number

    local newNum = ""
    for i=num:len(), 1, -1 do
        digit = tonumber(num:sub(i))

        printLog("In round(num="..origNum..",places="..places.."), i="..i..", num="..num..", newNum="..newNum..", digit="..digit..".")

        if digit == 9 then
            if i > 1 then
                newNum = "0"..newNum
            else
                newNum = "10"..newNum
            end
        elseif digit == nil then
            newNum = "."..newNum
        else
            if i > 1 then
                newNum = num:sub(1,i-1)..(digit + 1)..newNum
            else
                newNum = (digit + 1)..newNum
            end

            printLog("Called as round(num="..origNum..",places="..places.."), returns \""..newNum.."\"")
            return tonumber(newNum) --No more 9s found, so we are done incrementing. Copy remaining digits, then return number.
        end -- if digit == 9 then
    end -- for i=num:len(), 1, -1 do

    printLog("Called as round(num="..origNum..",places="..places.."), returns \""..newNum.."\"")

    return tonumber(newNum)
end -- function round(num, places)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant