#openobserve-cloud

Discrepancy in Histogram Counts on OpenObserve

TLDR rito reported incorrect counts on the OpenObserve histogram. Hengfei explained this might be due to batch ingestion or lack of '_timestamp'; the former makes the server ingest logs rapidly while the latter causes high volume in a small time range. rito acknowledged the advice.

Powered by Struct AI
6
5mo
Solved
Join the chat
Jun 30, 2023 (5 months ago)
rito
Photo of md5-a462bbe4adef925469661ad5969086c5
rito
06:18 AM
I sent the access logs of my server to OpenObserve. It is well remembered. Excellent. 👍
However, the counts on the histogram may not be correct. It does not match the log count. This is not a critical issue for me, but I will report it.
Image 1 for I sent the access logs of my server to OpenObserve. It is well remembered. Excellent. :+1:
However, the counts on the histogram may not be correct. It does not match the log count. This is not a critical issue for me, but I will report it.
Hengfei
Photo of md5-c30bb074b7d997d2cd6e689678b65dc1
Hengfei
06:19 AM
How do you think it is wrong?
rito
Photo of md5-a462bbe4adef925469661ad5969086c5
rito
06:21 AM
It looks like accesses are spiking, but they are not actually spiking. Also, the log count is not this high.
Hengfei
Photo of md5-c30bb074b7d997d2cd6e689678b65dc1
Hengfei
06:25 AM
if you use fluent-bit or vector, those tools have cache and batch ingestion. when you suddenly connect to OpenObserve it will ingest everything as quickly, after ingesting everything in the cache, you can see the speed reduce to almost same as your log produce speed.
06:27
Hengfei
06:27 AM
And if you don't provide _timestamp yourself, we will use the ingest time as _timetamp, it will looks like your screenshot, in a small time range, it has high volume, but after minutes it reduced.
rito
Photo of md5-a462bbe4adef925469661ad5969086c5
rito
06:29 AM
I did not write down the usage. I am sending logs using http requests.

Thanks for the info I will try sending it with _timestamp.