However, Prometheus is limited by its technical architecture design and has performance bottlenecks in processing massive amounts of data. The following problems may occur:
• Single-threaded processing: PromQL calculations are single-threaded, which may cause performance bottlenecks under high load, especially when querying time ranges where all blocks are covered.
• Data persistence issue: To avoid excessive data volume after long-term operation, the TSDB storage layer will delete some blocked data after a certain period of time, and the long-term persistence of the data cannot be guaranteed .
As an observability base, openGemini is compatible with Prometheus API and operators, which can give full play to its technical advantages and further bring more efficient and low-cost products in the scenario of massive indicator data analysis. It has many technical advantages, such as:
1. Elastic and scalable, supporting hundreds of nodes and PB- level data storage
2. Column storage, using dedicated compression algorithms japan mobile phone number list for different data types
3. Flexible deployment, compilation without external dependencies
4. Sub-second data query response, vectorized execution engine, and multiple computing optimizations
5. Most applications can be seamlessly connected
Before introducing the key optimization techniques for executing promql in OpenGemini, let us first understand the overall framework and data partitioning method of OpenGemini.
The overall framework of openGemini is mainly divided into four parts, including supported interface protocols. The green part is the already supported protocol, such as the data row write protocol and influxql query. The yellow part is the part added and optimized in the new version. The interface layer includes the newly compatible prom write API and promql for query. The computing layer mainly parses query statements and generates execution plans, including the existing parsing and optimizer, vectorized execution engine, and the implementation of various physical operators.