Discussing SIMD Images for Intel Servers Stability and Performance
TLDR The authors discussed the simd images by Prabhat for the on-prem intel servers by Chris. Chris conducted some tests with some issues. Prabhat suggested benefits are expected in aggregation queries.
Prabhat
Thu, 20 Jul 2023 12:26:29 UTCYou can download the simd images from
Prabhat
Thu, 20 Jul 2023 12:27:09 UTCWe have not done a comparative performance benchmarking of simd vs basic yet.
Chris
Thu, 20 Jul 2023 12:30:43 UTCah, good, will give that a try when i get a chance.... is the simd version regarded as stable?
Prabhat
Thu, 20 Jul 2023 17:55:45 UTCYes, we use it a lot.
Chris
Fri, 21 Jul 2023 14:57:20 UTCi pulled the amd64-simd-0.5.0 image, extracted the executable, and ran that on our test standalone instance (non-containerised).... it worked fine, but... repeatedly running a query over 56M records scansize 147098MB reported 700-800ms for the non-simd exec and 800-900ms for the simd one... admittedly a very rough-and-ready test but i wasn't expecting to see it worsen.... have i missed anything obvious?
Prabhat
Fri, 21 Jul 2023 16:46:18 UTCHow have been results on aggregations? Dashboards?
Chris
Fri, 21 Jul 2023 16:49:15 UTCi haven't tried much else.... that's why i was curious as to which functions you think it would benefit and which it probably wouldn't.... assume there is no config or other packages etc necessary - just the binary?
Prabhat
Fri, 21 Jul 2023 16:49:57 UTCIt should be just the binary. There is no benefit expected in search. However we expect big benefits in aggregation queries.
Chris
Fri, 21 Jul 2023 16:50:50 UTCok.... will bear that in mind and do some more tests when i get time (end of the day here now)
Prabhat
Fri, 21 Jul 2023 16:51:48 UTCWhat is configuration of the machine on which you tested this?
Chris
Fri, 21 Jul 2023 16:52:46 UTCsorry, people waiting for me... will send you details next week
Prabhat
Fri, 21 Jul 2023 16:52:57 UTCno worries. TTYL
OpenObserve
Indexed 419 threads
OpenObserve is an open-source, petabyte-scale observability platform for the cloud native realm, offering a 10x cost reduction and 140x less storage use compared to competitors like Elasticsearch or Splunk. Built in Rust for exceptional performance, it offers comprehensive features like logs, metrics, traces, dashboards, and more | Knowledge Base powered by Struct.AI
View in App
Chris
Thu, 20 Jul 2023 11:45:46 UTCour on-prem intel servers all support simd/avx512.... is that going to be released as a binary linux-amd64 release? and do you have any indicative figures of how much is gained during which operations?