Elasticsearch custom scoring and script scoring

2024-07-12

In Elasticsearch, the default scoring mechanism (such as the BM25 algorithm) can provide good search result ranking in most cases. However, in some specific scenarios, we may need to sort the search results more finely according to business needs. In this case, Elasticsearch provides custom scoring (Function Score Query) and script scoring (Script Score) functions to meet these complex needs. This article will introduce in detail how to customize scoring and script scoring in Elasticsearch.

1. Custom Scoring (Function Score Query)

1.1 Definition and Principle

Custom scoring allows us to modify the default score of each document based on a set of predefined functions. These functions can calculate new scores based on the document's field values, query conditions, or other factors. Ultimately, Elasticsearch will combine the original score with the custom score to determine the order of the search results.

1.2 Common functions

Elasticsearch provides a variety of built-in functions for custom scoring, including but not limited to:

weight: Assign a uniform weight to all matching documents.
field_value_factor: Calculates the score based on the field values in the document.
random_score: Generates a random score for a document.
decay_function: The decay score is calculated based on the distance between the document and a certain point (such as geographic location, time, etc.). Common decay functions include Gaussian, linear, and exponential.
script_score: Use a custom script to calculate the score.

1.3 Examples

Suppose we have an e-commerce website and users want to consider both the price and rating of a product when searching for it. We can usefunction_scoreQuery to achieve this requirement:

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "name": "手机"
        }
      },
      "functions": [
        {
          "field_value_factor": {
            "field": "price",
            "modifier": "reciprocal",
            "params": {
              "scale": 100
            }
          }
        },
        {
          "field_value_factor": {
            "field": "rating",
            "missing": 1.0
          }
        }
      ],
      "score_mode": "sum",
      "boost_mode": "replace"
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

In this example, we usefield_value_factorThe function calculates a custom rating based on the product's price and rating. The price uses inverse decay (reciprocal), so that the lower the price, the higher the score; the score directly uses the field value. Finally, we add the scores of the two functions and replace the original score.

2. Script Score

2.1 Definition and Principle

Scripted scoring allows us to use a custom scripting language (such as Painless) to calculate the score of a document. This approach provides maximum flexibility and can calculate the score according to almost any logic.

2.2 Examples

Suppose we want to adjust the rating based on the inventory quantity of the product, so that the less inventory the product has, the higher the rating. We can usescript_scoreFunction to achieve this requirement:

GET /products/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "name": "手机"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "source": "Math.max(0, 1 - doc['stock'].value / 100.0)",
              "lang": "painless"
            }
          }
        }
      ],
      "score_mode": "multiply",
      "boost_mode": "replace"
    }
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

In this example, we use the Painless scripting language to calculate the rating. The script calculates the rating based on the inventory quantity of the product (doc['stock'].value) calculates a score where more inventory means a lower score and fewer inventory means a higher score. We then multiply this score with the original rating to adjust the final search result ranking.

Conclusion

Custom scoring and script scoring are powerful features provided by Elasticsearch, which allow us to fine-tune the sorting of search results according to business needs. By using these features properly, we can improve the user experience and meet the diverse needs of users. However, it should be noted that custom scoring and script scoring may increase the complexity and computational cost of queries, so in actual applications, it is necessary to weigh the pros and cons and use them with caution.

Technology Sharing