Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output not compatible with the GKE integration #5

Open
ricmalta opened this issue May 15, 2018 · 3 comments · May be fixed by #7
Open

Output not compatible with the GKE integration #5

ricmalta opened this issue May 15, 2018 · 3 comments · May be fixed by #7

Comments

@ricmalta
Copy link

ricmalta commented May 15, 2018

I'm doing a basic integration with negroni and logrus middleware using this formatter
. However the format is not being recognised by stackdriver. My Implementation:

utils.logger.go

package utils

import (
	"os"

	stackdriver "github.com/TV4/logrus-stackdriver-formatter"
	"github.com/sirupsen/logrus"
)

// NewLogger instance
func NewLogger(name string) (*logrus.Logger, error) {
	logger := logrus.New()
	logger.Out = os.Stdout
	logger.Formatter = stackdriver.NewFormatter(
		stackdriver.WithService(name),
		stackdriver.WithVersion("v0.1.0"),
	)

	return logger, nil
}

Middleware implementation

// NewMainMiddleware creates a middleware with common app middlewares
func NewMainMiddleware(middleware *negroni.Negroni, apiConfig config.APIConfig) *negroni.Negroni {
	log, _ := utils.NewLogger("http")

	return middleware.
		With(negroni.NewRecovery()).
		With(negronilogrus.NewMiddlewareFromLogger(log, "http")).
		With(LanguageMiddleware{}).
		With(timeoutMiddleware(apiConfig))
}

Output sample

{"timestamp":"2018-05-15T12:50:47Z","message":"completed handling request","severity":"INFO","context":{"data":{"measure#http.latency":193507,"method":"GET","remote":"10.132.0.2:60088","request":"/v1/healthz","status":200,"text_status":"OK","took":193507}}}
{"timestamp":"2018-05-15T12:50:51Z","message":"started handling request","severity":"INFO","context":{"data":{"method":"GET","remote":"10.132.0.2:60140","request":"/v1/healthz"}}}
{"timestamp":"2018-05-15T12:50:51Z","message":"completed handling request","severity":"INFO","context":{"data":{"measure#http.latency":70072451,"method":"GET","remote":"10.132.0.2:60140","request":"/v1/healthz","status":200,"text_status":"OK","took":70072451}}}```

Stackdriver output

{
 insertId:  "1vpqxs3g2enyp0w"  
 jsonPayload: {
  context: {
   data: {
    measure#http.latency:  164545     
    method:  "GET"     
    remote:  "*******:*****"     
    request:  "/v1/healthz"     
    status:  200     
    text_status:  "OK"     
    took:  164545     
   }
  }
  message:  "completed handling request"   
  timestamp:  "2018-05-15T12:45:41Z"   
 }
 labels: {
  compute.googleapis.com/resource_name:  "fluentd-gcp-v2.0.9-zv825"   
  container.googleapis.com/namespace_name:  "*******-staging"   
  container.googleapis.com/pod_name:  "api-*******-service-api-********-*****"   
  container.googleapis.com/stream:  "stdout"   
 }
 logName:  "projects/********-*****/logs/*******-service-api"  
 receiveTimestamp:  "2018-05-15T12:45:44.697156895Z"  
 resource: {
  labels: {
   cluster_name:  "*******-gke"    
   container_name:  "*******-service-api"    
   instance_id:  "*****************"    
   namespace_id:  "*******-staging"    
   pod_id:  "*******-service-api-********-*****"    
   project_id:  "**********"    
   zone:  "*********"    
  }
  type:  "container"   
 }
 severity:  "INFO"  
 timestamp:  "2018-05-15T12:45:41Z"  
}

As you can see the playload is in the jsonPayload.context.data and not being actually filtered by stackdriver. By your code, the fields are not completely compatible with the current (v2) LogEntry definition: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry.

There is any easy solution to this or this module is deprecated?

@icco
Copy link

icco commented Dec 28, 2018

I'm also seeing this issue. @jsageryd or @marcusolsson, any thoughts on this?

@ricmalta maybe make a setting to switch between v1 and v2 formatting?

@icco icco linked a pull request Dec 29, 2018 that will close this issue
@marcusolsson
Copy link
Contributor

I'm no longer a maintainer of this repository. Is this still being maintained, @jsageryd?

@jsageryd
Copy link
Contributor

I am not maintaining it. Perhaps @gubyb or @wileeam would know?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants