Understanding and Integrating with OpenObserve

TLDR 刘希晨 sought clarification on the difference between ZincSearch, ZincObserve, and OpenObserve. Prabhat responded and actively engaged in a long discussion regarding an integration for 刘希晨's microservice framework. Hengfei shared that the issue 刘希晨 faced with traces upload was acknowledged and fixed.

Photo of 刘希晨
刘希晨
Fri, 07 Jul 2023 16:26:57 UTC

hi, I am new to openobserve here, could anyone explain the difference between ZincSearch, ZincObserve, OpenObserve for me? I didn't get a lot of messages from google about this.

Photo of Prabhat
Prabhat
Fri, 07 Jul 2023 18:37:23 UTC

1. OpenObserve - The awesome Observability tool that allows you to store and analyze logs, metrics, traces with additional functionality of dashboards and functions built with multi-tenancy in mind. It can scale to hundreds of nodes and work with petabyte scale data. Log search uses SQL, partitioning, brute force and caching to be highly efficient. Majority of our development effort is focused on this project. 2. ZincObserve - The old name of OpenObserve. ZincObserve was renamed to OpenObserve in June 2023 3. ZincSearch - An open source tool built initially by our team that is written in go. It is a full text search engine that utilizes inverted indexes for indexing and searching. It is a single node search engine that many people are using in their products. We do not recommend you to use this for log search but only for app search where a single node can suffice your needs. It's in maintenance mode and we are not actively adding new features to it. Hope this helps.

Photo of 刘希晨
刘希晨
Sat, 08 Jul 2023 01:34:32 UTC

Thanks very much for your answer, I would like to integrate OpenObserve into my own microservice framework, and I noticed it's still in alpha, is there any plan on GA schedule?

Photo of Prabhat
Prabhat
Sat, 08 Jul 2023 01:36:06 UTC

What features of OpenObserve would you want to use?

Photo of 刘希晨
刘希晨
Sat, 08 Jul 2023 05:01:52 UTC

I am creating my own microservice framework, it includes gateway, discovery and network infrastructure help building RPC based services, and I have been looking for a infrastructure that could collect logs, metrics, and traces, then I met OpenObserve which totally fits my idea. In my perspective, the log framework which generates logs, the gateway and RPC component which generates traces, the JVM monitor which generated metrics, are all written by myself,and I can fully control the data generation side, then I start searching for infrastructures that could handle the store and analyze part for me, the major concerns are:

Photo of 刘希晨
刘希晨
Sat, 08 Jul 2023 05:18:22 UTC

1. Want to be simple for enterprise to deploy and expand 2. Can be simply integrated into my framework, I don't want to use some offical-client to do the data-push jobs, because I want to put all the network logic inside the RPC part, so I prefer to use HTTP based collecting strategy 3. Powerful UI visualization would be great for me At first I am doing research about the ELK stack, and I think it might be too heavy and complex for my requirements, then I met with OpenObserve and I think it's a great fit. Do you agree with me? Excuse me for my poor English, Thank you guys for replying!

Photo of Prabhat
Prabhat
Sat, 08 Jul 2023 05:24:33 UTC

I think you have all the right understanding and OpenObserve looks to be a great fit for your requirements. We have many hundreds of deployments of OpenObserve in production. You can consider logs and traces to be GA. If you encounter any issues do let us know. Metrics should become GA in 2-4 months. OpenObserve does fit all of your needs in terms of : 1. easy to deploy in an enterprise setup 2. HTTP APIs for ingestion 3. Great visualization and it is super lightweight compared to Elasticsearch. I agree with your assessment, and your english is fine. Do let us know if you need any help implementing OpenObserve.

Photo of 刘希晨
刘希晨
Sat, 08 Jul 2023 06:26:19 UTC

Great, thanks for your replying! Currently I am building the database access part of my framework, after that I will start coding about the metrics part, I will let you guys know if I encounter any issure using OpenObserve! I hope both my project and OpenObserve will achieve success in the future!

Photo of 刘希晨
刘希晨
Sun, 09 Jul 2023 19:10:19 UTC

Hi, I started playing with OpenObserve successfully with uploading logs from the tutorial, then I tried uploading traces with error, I am calling `` with basic authentication and body as json format using `` the example json from the opentelemetry github repo, I think it should be fine, the console give me this error: ```thread 'actix-server worker 2' panicked at 'called `Result::unwrap()` on an `Err` value: Error("EOF while parsing an object", line: 1, column: 1)', src\service\traces\``` Is there any mistakes in my procedure?

Photo of Prabhat
Prabhat
Sun, 09 Jul 2023 19:13:32 UTC

We have a bug for this that needs resolution -

Photo of Hengfei
Hengfei
Mon, 10 Jul 2023 02:43:51 UTC

刘希晨there is a bug and we already fixed, but haven't make new release.

Photo of 刘希晨
刘希晨
Mon, 10 Jul 2023 03:27:04 UTC

Thanks, I will wait for the new release and then do some tests around:+1: