Querying Local Storage Files and Ingesting Logs with OpenObserve
TLDR Rinshad asks about querying local storage files and ingesting logs. Prabhat suggests using parquet file format and tools like vector and fluentbit. Mark mentions Sublime's plugin for editing parquet files.
Jun 20, 2023 (5 months ago)
Rinshad
03:31 PMPrabhat
03:35 PMMark
09:01 PMJun 21, 2023 (5 months ago)
Rinshad
05:27 AMPrabhat
01:51 PMRinshad
02:51 PMRinshad
02:53 PMPrabhat
04:03 PMJun 22, 2023 (5 months ago)
Rinshad
10:46 AMPrabhat
12:30 PMPrabhat
12:38 PM/var/lib/postgresql/<version>/main/pg_log/postgresql-*.log
OpenObserve
Indexed 406 threads (74% resolved)
Similar Threads
Troubleshooting Openobserve Setup with Azure Blob on K8S
David had difficulties setting up Openobserve with Azure Blob in Kubernetes. Prabhat provided guidance and setup values, informing David about possible data transmission delays, which resolved the issue.
Troubleshooting Openobserve Issue On Linux Machine
Sushma encountered difficulties viewing pods and creating multiple streams in Openobserve. Prabhat and Hengfei suggested upgrading the software, adjusting SQL syntax, and amending the configuration setup. Despite progress, the issue remains partially unresolved.
Troubleshooting Kubernetes Deployment and PostgreSQL Connection
Johnson faced issues fetching summary from a main page of a Kubernetes deployment, also observed PostgreSQL connection error on Azure, while the same setup was successful in AWS. Prabhat and Hengfei explored solutions but the problems persisted.
Resolving json logs interpretation issue in OpenObserve with Vector
Gabriel was facing a problem with OpenObserve interpreting json logs as plain text, using Vector for log routing. Ashish helped by suggesting adding a json parser to the Vector configuration, which solved the issue.
Troubleshooting Kafka Log Setup with OpenObserve
Ivan encountered an issue while setting up Kafka logs with OpenObserve and shared their vector config. Prabhat suggested a possible resource issue, but a final resolution wasn't determined.