Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New lines are escaped now #608

Closed
pkopac opened this issue Jul 31, 2017 · 28 comments
Closed

New lines are escaped now #608

pkopac opened this issue Jul 31, 2017 · 28 comments

Comments

@pkopac
Copy link

pkopac commented Jul 31, 2017

Hi!
With a recent update I've noticed that my debug data dumps in logs get fully escaped. Previously, I had nicely formatted data structs in output (using spew.Sdump), so I could actually read them.

Code:

  • data is a complex data structure with arrays, pointers, many custom types, optional fields etc.
    log.Debug(spew.Sdump(data))

And now I get:
(*pkg.Data)(0xc423e8cb00)({\n UUID: (string) \"\",\n Property:...

I've browsed through recent updates, but can't find a way to turn this off other then fixing version of logrus to some previous one. Please advise. The data dumps are unreadable now.

@dmathieu
Copy link
Contributor

This is probably because we hardened the string escaping with %q instead of the previous unsafe escaping.
See #580

I'm afraid there isn't much we can do here. This could actually be considered as a bug which has been fixed. Being able to send new lines in your log could make them harder to read, and actually reduce the purpose of structured logging, where you would only log specific meaningful fields not dump a whole json object.

@JelteF
Copy link

JelteF commented Sep 7, 2017

This makes sense when logging in production with an elk stack that parses the message, but when logging for development this is really annoying. Stack traces that are logged are now unreadable. If logged to TTY could this change be undone (just like we have colouring in this case? Or maybe make it configurable, because sometimes when logging to a file for development you also want the previous behaviour.

@berniedurfee-ge
Copy link

Yes please on this one. Having a switch of some sort would be very nice.

@himanshug
Copy link

Another use case I have where I am printing json serialized objects in the debug logs. In the printed json, all double quotes are escaped too making it totally unreadable.

@reefbarman
Copy link

we have run into the need to print the stack traces in our logs as well and the removal of the new lines on stdout makes them very hard to read, is this going to be considered?

@hmalphettes
Copy link

For human consumption, I am using https://github.com/antonfisher/nested-logrus-formatter which does not escape the text content.

For example:

Mar 18 09:32:26.740 [INFO]  [Content:## Generated by my-tool
# Read the policy
path "sys/policies/acl/a-policy" {
  "capabilities" = [ "read" ]
}

path "secret/*" {
  "capabilities" = ["read","list"]
}
path "auth/token/create/a-role" {
  "capabilities" = ["update"]
  "denied_parameters" = {"*"=[]}
}
] [Environment:dev]

I hope this helps.

@nzt4567
Copy link

nzt4567 commented Apr 16, 2019

A switch would definitely be handy...

For example LaTeX can use \n as a start of custom command very often which makes it quite hard to convert these escaped new lines back to, you know, just new lines. And debugging dynamically generated LaTeX templates without being able to actually look at the generated template is, you know, just sad.

@Koshmaar
Copy link

Another vote for being able to decide whether the log should escape or not-escape newlines. Come on, this is common sense.

@naughtyGitCat
Copy link

Why logrus keep line number and new line escape so unconvenient? This is golang way?

@ParadiseDS
Copy link

Vote for an option to decide if newline should be escaped in log.
Newline is really more readable when log stack trace.

@freeformz
Copy link
Collaborator

freeformz commented Oct 17, 2019

To be clear this issue is asking for an option that produces this:

time="2009-11-10T23:00:00Z" level=info traceField=line 1
line 2
line 3 otherLogField=bar

^ assuming a trace that is:

line 1
line 2
line 3

@aido93
Copy link

aido93 commented Dec 12, 2019

I managed to do that approximately in this way:

    if log.GetLevel()==log.DebugLevel {
        strSlice:=strings.Split(str, "\n")
        for _, s := range strSlice {
            log.Debug(s)
        }
    }

It's not ideal solution but at least it works somehow... Also it will be called only in development when you set level to DebugLevel. I think there is no need to use debug messages in production. So no need to use multiline output in production. Hope, this code helps someone;)

@stale
Copy link

stale bot commented Feb 26, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Feb 26, 2020
@krasoffski
Copy link

Need ability to enable or disable such behavior and ability to configure striping/replacing new lines in message fields.

@stale stale bot removed the stale label Feb 27, 2020
@nous23
Copy link

nous23 commented Apr 22, 2020

reasonale to enable newline...

@Frug
Copy link

Frug commented May 1, 2020

For all kinds of local testing, this option would be extremely useful.

@stale
Copy link

stale bot commented Jun 30, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@mieubrisse
Copy link

Re-pinging on this to prevent it from getting closed. Right now, the only option that I can tell is fmt.Fprintf(logrus.StandardLogger().Out, theStr)

@stale stale bot removed the stale label Jul 4, 2020
@bmon
Copy link

bmon commented Aug 27, 2020

We solved this with a custom formatter that looks for traces in a specific field and appends them to the log. Replace stack with your field of choice.

func TestMain(m *testing.M) {
        testLogger.SetLevel(logrus.WarnLevel)
        testLogger.SetFormatter(&StackFormatter{})
        os.Exit(m.Run())
}

type StackFormatter struct {
        logrus.TextFormatter
}

// Render a log entry with a TextFormatter - except if there is a `stack` field, print it (with newlines) after the log message.
func (f *StackFormatter) Format(entry *logrus.Entry) ([]byte, error) {
        stack, ok := entry.Data["stack"]
        if ok {
                delete(entry.Data, "stack")
        }
        res, err := f.TextFormatter.Format(entry)
        if stack, ok := stack.(string); ok && stack != "" {
                res = append(res, []byte("stack trace:\n"+stack)...)
        }
        return res, err
}

deads2k added a commit to deads2k/installer that referenced this issue Oct 5, 2020
… separate lines

sirupsen/logrus#608 indicates this is intentional, but
human readable conditions often use newlines to separate output for humans.
Workaround the problem for clusteroperator output.  The overall error from the installer
would also benefit, so those lines are tagged with TODOs.
abhinavdahiya added a commit to abhinavdahiya/installer that referenced this issue Oct 19, 2020
Currently the logrus hook runs formatter once for each entry, which causes newlines to be escaped see sirupsen/logrus#608
Because of this multiline messages end up looking like

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
same message multiline 1
LEVEL next messsage
```

With this change the hook will format each entry's message split on newlines so that,

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
LEVEL same message multiline 1
LEVEL next messsage
```

This will help 2 cases,
- openshift@af9b49c like informational messages that are require multiple lines to be more user-friendly.
- meesages from cluster operators on install-complete failure see openshift#4242
abhinavdahiya added a commit to abhinavdahiya/installer that referenced this issue Oct 19, 2020
Currently the logrus hook runs formatter once for each entry, which causes newlines to be escaped see sirupsen/logrus#608
Because of this multiline messages end up looking like

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
same message multiline 1
LEVEL next messsage
```

With this change the hook will format each entry's message split on newlines so that,

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
LEVEL same message multiline 1
LEVEL next messsage
```

This will help 2 cases,
- openshift@af9b49c like informational messages that are require multiple lines to be more user-friendly.
- meesages from cluster operators on install-complete failure see openshift#4242
@stale
Copy link

stale bot commented Oct 26, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Oct 26, 2020
@krasoffski
Copy link

It is not stale!

@stale stale bot removed the stale label Oct 26, 2020
@sunlintong
Copy link

same to me .俺也一样

@sunlintong
Copy link

I set ForceColors true to resolve this problem, BUT I don't know why

logrus.SetFormatter(&logrus.TextFormatter{
    ForceColors: true,
})

kwoodson pushed a commit to kwoodson/installer that referenced this issue Nov 30, 2020
Currently the logrus hook runs formatter once for each entry, which causes newlines to be escaped see sirupsen/logrus#608
Because of this multiline messages end up looking like

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
same message multiline 1
LEVEL next messsage
```

With this change the hook will format each entry's message split on newlines so that,

```go
logrus.Info(`some message
same mesage multiline 1`)
logrus.Info("next meesage")
```

```
LEVEL some message
LEVEL same message multiline 1
LEVEL next messsage
```

This will help 2 cases,
- openshift@af9b49c like informational messages that are require multiple lines to be more user-friendly.
- meesages from cluster operators on install-complete failure see openshift#4242
@yy4ever
Copy link

yy4ever commented Dec 15, 2020

logger.SetFormatter(&logrus.TextFormatter{
    DisableQuote: true,
})

c7455de

amnk added a commit to amnk/terragrunt that referenced this issue Feb 2, 2021
Terragrunt integration tests depend on newlines, and to ensure that
logrus can properly print newlines with `\n` `DisableQuote: true` is
needed [1]. In particular, this fixes
`TestIncludeDirsDependencyConsistencyRegression` and
`TestIncludeDirsStrict`.

[1] - sirupsen/logrus#608
yorinasub17 pushed a commit to gruntwork-io/terragrunt that referenced this issue Feb 2, 2021
* Option for less verbose output

This moves majority of Terragrunt's output to DEBUG level, and sets
default output mode to ERROR. The idea is that in default run user should
see much less output.

Closes: #432

* Updated after review

Apart from small fixes, this:
- adds deprecation check for command line arguments, and gives a
  warning if `--terragrunt-debug` is used;
- imports `logrus` under its own name to avoid confusion;

Related: #432

* Updates after review | 2

Main change - global LogEntry object introduced in `util/logger.go`. It
helps us log properly when we didn't yet parse command line arguments

* Fix tests for `cli/cli_app.go`

P.S. Tests rock!

* Rework global logger

Global logger should be a fallback option, rather a way to go. So
instead of using the same everywhere, we use global if no other logger
is available (for example, if we have not yet parsed arguments).

This change also addresses other PR review comments, like:
- wrong documentation;
- using `logrus.WarnLevel.String()` instead of hardcoded strings;
- updated `isDeprecatedOption()`, with a more clear usage;

* Fix typo in docs section

* Apply `go fmt` properly

* Apply `go fmt` properly | 2

* Fix `TestParseTerragruntOptionsFromArgs` test

* Terragrunt should output all log to stderr

* Set stderr as default output

* Update TestApplySkipTrue()

Add `--terragrunt-log-level info`, so that we could test it.

* Fix tests

This fixes tests that rely on double output of `run_shell`, as well as
brings back `--terragrunt-debug`, but in slightly different form: to
only generate `terragrunt-debug.tfvars.json`

* Remove `--terragrunt-debug` from excluded and fix `\n`

Terragrunt integration tests depend on newlines, and to ensure that
logrus can properly print newlines with `\n` `DisableQuote: true` is
needed [1]. In particular, this fixes
`TestIncludeDirsDependencyConsistencyRegression` and
`TestIncludeDirsStrict`.

[1] - sirupsen/logrus#608

* Bring `--terragrunt-debug` tests back
@hdlinh1808
Copy link

logger.SetFormatter(&logrus.TextFormatter{
    DisableQuote: true,
})

c7455de

It's worked for me! Thanks

@github-actions
Copy link

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Sep 28, 2021
@github-actions
Copy link

This issue was closed because it has been inactive for 14 days since being marked as stale.

@dablelv
Copy link

dablelv commented Nov 15, 2022

为什么不提供一个开关,比如我把日志写入日志文件中,在查看日志的时候,换行缩进显示 JSON 格式的日志将更加可读。Why not provide a switch, for example I write the log to the log file, when viewing the log, it will be more readable to display the log in JSON format with newline indentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests