Issues with Vector Log Transmission to OpenObserve
TLDR Ivan was having trouble setting a dynamic sink url in Vector for OpenObserve. Chris suggested that he needed a separate vector sink for each org/stream as current Vector version didn't support dynamic construction of sink url.
Prabhat
Thu, 17 Aug 2023 10:34:02 UTCHaven't dealt with this yet. I checked the config and could not find something quickly. vector discord server has folks who will be more knowdgeable and should be able to help.
Chris
Thu, 17 Aug 2023 10:46:53 UTCWhen i looked a few weeks ago, vector doesn't support dynamic construction of the sink url. It is on their wish-list (can't remember whether they have committed to implementing it). But at the moment you need a separate vector sink for each org/stream
Ivan
Thu, 17 Aug 2023 10:48:38 UTCGot it, thanks for the replies
Chris
Thu, 17 Aug 2023 10:51:57 UTCi see vector-0.32.0 was released on 15 Aug... at a quick glance i don't see anything about this but haven't looked in detail yet
OpenObserve
Indexed 419 threads
OpenObserve is an open-source, petabyte-scale observability platform for the cloud native realm, offering a 10x cost reduction and 140x less storage use compared to competitors like Elasticsearch or Splunk. Built in Rust for exceptional performance, it offers comprehensive features like logs, metrics, traces, dashboards, and more | Knowledge Base powered by Struct.AI
View in App
Ivan
Thu, 17 Aug 2023 10:25:52 UTCHi, need advice, maybe this problem has already been solved, I am using vector to send logs to openobserve Config vector: ```sinks: openobserve: type: http inputs: - throttle_out uri: '{{"{{url}}"}}' method: post auth: strategy: basic user: ""
password: "Complexpass#123"
compression: gzip
encoding:
codec: json
timestamp_format: rfc3339
healthcheck:
enabled: false
transforms:
throttle_in:
inputs:
- msk
threshold: 1500
type: throttle
window_secs: 1
throttle_out:
inputs:
- create_index
threshold: 500
key_field: '{{ "{{" }} stream {{ "}}" }}'
type: throttle
window_secs: 1
create_index:
drop_on_abort: true
drop_on_error: true
inputs:
- throttle_in
source: |-
if exists(.) {
.app, err = downcase(.)
if err != null {
log("Downcase error: " + err, level: "error", rate_limit_secs: 60)
}
} else {
.app = "unallocated"
}
.url = "