TLDR sparrow inquired about pushing logs to S3 and installing ZincObserve using a Helm chart. Hengfei provided assistance and documentation, and Prabhat confirmed the availability of the Helm chart for S3.
Yes i’ve gone through this doc, but where exactly i have to mention in fluent-bit so that it will select s3 as a storage type, because by default it is setting up as disk storage
as you can see all index is using disk as a storage
oh sorry, you use zincSearch for logs?
zincSearch has some problems for support s3. we suggest use zincObserve for logs.
do we have any helm chart for zincObserve?
Prabhat are we ready for helm?
i think we are working for helm, should be ready this week.
If not helm then how I can install zincObserve?
single node, right?
you can use k8s deploy
Okay got it, Thanks. Do zincObserve also use telemetry to fetch data?
Yep, you can use `ZO_TELEMETRY=false` to disable it
just report some usage for us to analytics how many user. no private data.
you also can disable it.
there is a quick start video.
but i can see it’s fetching `querystring, match`
In our logs there so many data, we can’t just let this to be available for others
this is zincSearch, it report this, because we have old api & new api, we want to know if we should delete old api.
Yes of course, you can disable it with the environment.
after you disable it, it will report nothing.
but in our production account we have blocked all the outbounds rules, in this case it would fail
i didnot get your point. which will fail? you already disable it.
is there anyway by which I can confirm the telemetry is disabled or not?
we provider an environment for disable it. if you want to confirm that, you can disconnect internet, itf it still report something, you will see network error.
you can try this with keep enable it. you will see the error.
After installing zincObserve from where sould i get the credential to login?
Also I’m not able to exec into the zincobserve pod neither with sh nor with bash
for production image, no, we use micro image. if you want login into the container, you can dev image.
where sould i get the credential to login?
you can see the deployment, it set the root user.
and in the document, it also mentioned the user name & password.
Helm chart should be ready this week
What is this error?
Hengfei
i have no idea, haven’t met this
Have you ingest some data?
yes
How
at first i was able to see the logs, but suddenly getting this error
Using fluent-bit
You can see
Yep
can you give some logs about the error in container for zincobserve
helm charts for s3 is now available -
Hello Hengfei I’ve a update on the above error
Which error? maybe i didn’t get it.
The above one that i reported
this one
Okay, if you have more logs about it on the server side?
i am not sure the problem.
how do you install zincobserve and the configure variables.
Actually i’m getting this error when I’m trying to run some query and it’s not returning anything
I used this manifest
```apiVersion: v1
kind: Service
metadata:
name: zincobserve
spec:
clusterIP: None
selector:
app: zincobserve
ports:
- name: http
port: 5080
targetPort: 5080
---
# create statefulset
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zincobserve
labels:
name: zincobserve
spec:
serviceName: zincobserve
replicas: 1
selector:
matchLabels:
name: zincobserve
app: zincobserve
template:
metadata:
labels:
name: zincobserve
app: zincobserve
spec:
securityContext:
fsGroup: 2000
runAsUser: 10000
runAsGroup: 3000
runAsNonRoot: true
# terminationGracePeriodSeconds: 0
containers:
- name: zincobserve
image: public.ecr.aws/zinclabs/zincobserve:latest
env:
- name: ZO_ROOT_USER_EMAIL
value:
okay, can you give some errors in the container
to check what happens with zincobserve
can you send me the logs of the container
Or, can you delete the deploy and try again, to see it there still have this error, if you can reproduce the error, it will be better.
let’s talk in private. i will ping you.
sparrow
Thu, 23 Feb 2023 07:29:20 UTCHello Team, I was exploring the product and able to install and and use it properly but I can’t find any way around by which i can push my logs to s3 bucket using fluent bit could you please help me