OTel Metrics Setup - 404 Error and Authentication Issues

TLDR Ioannis encountered a 404 error and asked for assistance. Prabhat confirmed a bug, which was later fixed by Patel. Assen also found discrepancies in the documentation, which Prabhat corrected.

Photo of Ioannis
Ioannis
Thu, 08 Jun 2023 19:24:28 UTC

Hey folks. I followed the guide on how to setup OTel for metrics, but I am getting a 404 each time the exporter hits the endpoint - any idea why?

Photo of Ioannis
Ioannis
Thu, 08 Jun 2023 19:25:32 UTC

for reference the setup is

Photo of Ioannis
Ioannis
Thu, 08 Jun 2023 19:26:00 UTC

```exporters: prometheusremotewrite: endpoint: headers: Authorization: Basic XXXXXXXXXXXXX==```

Photo of Ioannis
Ioannis
Thu, 08 Jun 2023 19:26:15 UTC

replacing sensitive values ofc :slightly_smiling_face:

Photo of Ioannis
Ioannis
Thu, 08 Jun 2023 19:54:05 UTC

in all actual honesty.. doesn't work either :slightly_smiling_face:

Photo of Hengfei
Hengfei
Fri, 09 Jun 2023 00:35:07 UTC

Thanks you for report this, we will quick fix it

Photo of Ioannis
Ioannis
Fri, 09 Jun 2023 06:29:58 UTC

Hengfei no worries. I guess I can try the container in the meantime? Also, who can answer some questions about pricing?

Photo of Hengfei
Hengfei
Fri, 09 Jun 2023 06:45:47 UTC

Prabhat Let you answer the questions about pricing

Photo of Prabhat
Prabhat
Fri, 09 Jun 2023 06:47:24 UTC

> in all actual honesty.. doesn't work either :slightly_smiling_face: Let me DM you

Photo of Ioannis
Ioannis
Fri, 09 Jun 2023 06:47:41 UTC

sure

Photo of Assen
Assen
Sun, 11 Jun 2023 13:33:47 UTC

I am also trying to make this work. specifies another endpoint - it always returns 404: `` The endpoint in this thread `` doesn't like the basic authorisation: {\"Error\":\"Authentication failed - invalid token: token contains an invalid number of segments\"}

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:41:12 UTC

Assen You may have an @ or : symbol possibly in the token. Try regenerating the token.

Photo of Assen
Assen
Sun, 11 Jun 2023 13:41:53 UTC

my user name is my email address, so yes, there's an @

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:42:07 UTC

yeah, apart from that

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:43:06 UTC

Are you using one of the code examples for opentelemetry from

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:43:56 UTC

Actually you should not have any no-numeric characters in ingestion token

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:44:08 UTC

we made sure of that when generating the token

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:44:20 UTC

unless it's not working well 100% of the time

Photo of Assen
Assen
Sun, 11 Jun 2023 13:50:17 UTC

Thanks! I am configuring an OTLP collector - no code - but your link helped. it brought me to which shows: TRACES / OTLP ```HTTP Endpoint: Authorization: Basic YX..``` and I get this error with the token from that page

Photo of Assen
Assen
Sun, 11 Jun 2023 13:50:50 UTC

I am trying to send metrics, no traces BTW

Photo of Assen
Assen
Sun, 11 Jun 2023 13:52:24 UTC

```exporters: prometheusremotewrite: endpoint: "" headers: Authorization: Basic YXNzZW4ua......```

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:53:07 UTC

is this working?

Photo of Assen
Assen
Sun, 11 Jun 2023 13:54:12 UTC

no, ``` "error": "Permanent error: Permanent error: remote write returned HTTP status 401 Unauthorized; err = %!w(<nil>): {\"Error\":\"Authentication failed - invalid token: token contains an invalid number of segments\"}"```

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 13:55:00 UTC

Let me check in my config

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 14:10:40 UTC

I can confirm that we have a problem. The URL and config are correct, but its not working due to a bug. The problem is only in cloud deployment. Open source version works well. Will fix it soon.

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 14:11:14 UTC

Will confirm it here once fixed.

Photo of Assen
Assen
Sun, 11 Jun 2023 14:13:20 UTC

Thanks for checking!

Photo of Patel
Patel
Sun, 11 Jun 2023 14:49:20 UTC

Assen Can you please check now? it should work now

Photo of Assen
Assen
Sun, 11 Jun 2023 17:53:43 UTC

that was amazingly fast! Thanks a lot Patel!

Photo of Assen
Assen
Sun, 11 Jun 2023 18:29:59 UTC

It works for me, but it would be useful to change to at

Photo of Prabhat
Prabhat
Sun, 11 Jun 2023 19:15:45 UTC

Thanks for pointing the issue. Fixed it.