Transition from sled DB to SQLite: A Discussion

TLDR Benjamin asked about the reasons for transition from sled DB to SQLite. Prabhat clarified that SQLite is less resource-hungry and simplifies metadata storage.

Photo of Benjamin
Mon, 18 Sep 2023 01:32:22 UTC

Hello! I noticed on the release page “Migrated from sled DB to sqlite for local mode (Auto upgrade included).” I’d be curious to know more about that - was there a stability issue with sled? Is SQLite faster? Something else? No real reason I’m asking, just curious. Gonna try to add tracing with OpenObserve to ! Wish me luck :crossed_fingers:

Photo of Prabhat
Mon, 18 Sep 2023 08:33:00 UTC

Benjamin sled was more resource hungry in our tests consuming a lot more memory compared to sqlte. We also started support for postgres and mysql for storing file list. It made a lot more easier to have more things on sql for metadata storage too.

Photo of Benjamin
Mon, 18 Sep 2023 14:43:00 UTC

Got it, thanks!!