Observe in New Relic

For a Groundplex deployed on Docker, observe metrics in New Relic.

After configuring and deploying the OTEL collector and the Groundplex nodes, follow these steps to observe metrics in New Relic:

  1. Run some pipelines on the Groundplex. In a few minutes, pipeline execution logs and Snaplex node metrics are visible in the New Relic dashboard.
  2. To view pipeline execution information in New Relic:
    1. In the left navigation pane, click Logs
    2. Above the table of logs, click Add column and select log_summary:

    New Relic log summary

  3. By default the Metrics & Events link is not pinned to the navigation pane. To check the values on the Metrics page:
    1. In the Quick Find box, enter Metrics & Events.
    2. Select the plexnode metric to view:
    New Relic Metrics and Events

Refer to Node metric reference for metric descriptions.

Troubleshoot issues

If New Relic does not receive data, check the OTEL collector logs. From the otel directory, run the logs command using the container ID: 
docker logs <open telemetry container id> -f
A normal log contains timestamps and metric values. If the New Relic API key isn't valid, the log contains the following: 
Value: 0.000000
	{"kind": "exporter", "data_type": "metrics", "name": "logging"}
2024-06-20T08:46:48.822Z	error	exporterhelper/queue_sender.go:101	Exporting failed. Dropping data.	{"kind": "exporter", "data_type": "metrics", "name": "otlphttp", "error": "not retryable error: Permanent error: rpc error: code = PermissionDenied desc = error exporting items, request to https://otlp.nr-data.net:4318/v1/metrics responded with HTTP Status Code 403, Message=, Details=[]", "dropped_items": 100}
go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1
	go.opentelemetry.io/collector/[email protected]/exporterhelper/queue_sender.go:101
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume
	go.opentelemetry.io/collector/[email protected]/internal/queue/bounded_memory_queue.go:52
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1
	go.opentelemetry.io/collector/[email protected]/internal/queue/consumers.go:43
2024-06-20T08:46:48.824Z	error	exporterhelper/queue_sender.go:101	Exporting failed. Dropping data.	{"kind": "exporter", "data_type": "metrics", "name": "otlphttp", "error": "not retryable error: Permanent error: rpc error: code = PermissionDenied desc = error exporting items, request to https://otlp.nr-data.net:4318/v1/metrics responded with HTTP Status Code 403, Message=, Details=[]", "dropped_items": 12}
go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1
	go.opentelemetry.io/collector/[email protected]/exporterhelper/queue_sender.go:101
go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume
	go.opentelemetry.io/collector/[email protected]/internal/queue/bounded_memory_queue.go:52
go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1
	go.opentelemetry.io/collector/[email protected]/internal/queue/consumers.go:43

Create a new API key and try again.