Troubleshooting VRL Function Query Run

TLDR Ted experiences OOM error while using VRL Function queries, leading to queries failing. Ashish helps to troubleshoot and suggests implementing VRL-functions during a narrower query time window. They also give specific codes to remove unused field names and in return Ted manages to resolve the error.

Photo of Ted
Ted
Tue, 31 Oct 2023 08:46:33 UTC

Hi, I just try VRL Function with dot ' . ' (see the pic) , querier pod got OOMKilled immediately. I install HA Deployment on k8s via helm chart without any resource limit configure. Is it issue of VRL function? Node pool: 2 CPU 16G Ram

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:47:24 UTC

are you trying function at query time?

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:47:59 UTC

if yes and if you are trying it on hugh amount of data..it can cause OOM

Photo of Ted
Ted
Tue, 31 Oct 2023 08:48:12 UTC

No, I just enter one dot. " . "

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:48:26 UTC

yes it can cause OOM

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:48:34 UTC

try removing .

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:48:40 UTC

and querying

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:48:57 UTC

what are trying to achieve with vrl?

Photo of Ted
Ted
Tue, 31 Oct 2023 08:51:31 UTC

I'm trying to learn VRL Function. I want to remove_unused_k8s_fileds like in pic (from ). But I can't find any example to do that. After that I want to using VRL function to parse my golang application log

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:52:34 UTC

what we suggest is use vrl functions during query time on small data ..by keeping time window to 1 min etc

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:53:05 UTC

del(._p) del(.kubernetes_container_hash) del(.kubernetes_docker_id) del(.kubernetes_pod_id) del(.kubernetes_labels_pod_template_hash) del(.kubernetes_annotations_checksum_luascripts) del(.kubernetes_annotations_checksum_config) .

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:53:31 UTC

you can try above code snippet with appropriate field names to remove unused fields

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:54:04 UTC

once you finzalize your function

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:54:21 UTC

associate it with steam so that function works at data ingestion

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:55:05 UTC

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:55:11 UTC

check this video

Photo of Ted
Ted
Tue, 31 Oct 2023 08:57:34 UTC

ok, I try with 1min frame and it's working now. Thank you! I just read docs at . Can I using this?

Photo of Ashish
Ashish
Tue, 31 Oct 2023 08:58:18 UTC

yes these should work..as long as you have appropriate data

Photo of Ted
Ted
Tue, 31 Oct 2023 08:59:42 UTC

Yay, thank you so much!

Photo of Ashish
Ashish
Tue, 31 Oct 2023 09:03:00 UTC

welcome

Photo of Ted
Ted
Tue, 31 Oct 2023 09:14:22 UTC

Hi, can I ask 1 more question. I want to parse this log to the table. What function do I need to parse it?

Photo of Ashish
Ashish
Tue, 31 Oct 2023 09:15:40 UTC

Photo of Ashish
Ashish
Tue, 31 Oct 2023 09:15:44 UTC

check if this helps

Photo of Ashish
Ashish
Tue, 31 Oct 2023 09:16:02 UTC

you can use

Photo of Ted
Ted
Tue, 31 Oct 2023 09:16:50 UTC

yes, thank you! :pray: