TLDR Mik inquired about the VRL function behavior in OpenObserve logs. Prabhat explained the system automatically flattens structures, a feature that won't change in the future.
Yes. OpenObserve flattens the structure with or without VRL
Thanks Prabhat
Prabhat just some inquiries: 1. May I know why? 2. Is this a behaviour we can disable? 3. Is this intended to be the case in the future? Thank you in advance! :pray:
data is stored in tabular format in parquet files and flattening is the best option. you can't disable it. it won't change . We understand that there can be inconveniences with this approach, but we think its a good tradeoff.
Thanks for the clarification. Appreciate it!
Mik
Mon, 02 Oct 2023 04:15:56 UTCHi everyone! Is this the expected behavior of the VRL function in the open observe logs? VRL function ```.log = parse_json!(.message_message) del(.message_message) .``` VRL playground output: ```{ "_timestamp": 1695956833809000, "id": "37821101218481254744296987650977054435587665849865928704", "log": { "applicationName": "sbc-outbound", "event": { "message": "Refresh RTP Engine connections: 10.100.12.93" }, "eventTime": 1695956829, "instanceName": "sip-little-woodpecker", "level": "info" }, "loggroup": "/log/sbc-sip/sbc-log", "logstream": "sip-little-woodpecker", ... "requestid": "11111xxxx-fe77-481c-b60b-73a33a1d9602", "subscriptionfilters": "[\"CloudwatchOpenobserveIngestStack-SubscriptionFilterXXXXX-wZGu0KhDhTIC\"]", "timestamp": 1695956833809 }``` OpenObserve logs: ```{ _timestamp:1695956113809000, id:37821085161944711802248324874853977452967836770446606336, log_applicationname:sbc-outbound, log_event_message:Refresh RTP Engine connections: 10.100.12.93, log_eventtime:1695956109, log_instancename:sip-little-woodpecker, log_level:info, "loggroup": "/log/sbc-sip/sbc-log", "logstream": "sip-little-woodpecker", ... "requestid": "11111xxxx-fe77-481c-b60b-73a33a1d9602", "subscriptionfilters": "[\"CloudwatchOpenobserveIngestStack-SubscriptionFilterXXXXX-wZGu0KhDhTIC\"]", "timestamp": 1695956833809 }``` OpenObserve will automatically append an `underscore` for nested JSON object keys?