๐Ÿง  ๊ฒ€์ƒ‰ ์ ์ˆ˜ ๋ถ„์„: explain ํ™œ์šฉํ•˜๊ธฐ

์ด ๋ฌธ์„œ๋Š” ?explain=true๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ฒ€์ƒ‰ ์ ์ˆ˜(BM25 ๋“ฑ)์˜ ๊ณ„์‚ฐ ๊ทผ๊ฑฐ๋ฅผ ํŒŒ์•…ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค.
์–ด๋–ค ๋ฌธ์„œ๊ฐ€ ์™œ ๋†’์€ ์ ์ˆ˜๋ฅผ ๋ฐ›์•˜๋Š”์ง€ ์ดํ•ดํ•˜๊ณ , ๊ฒ€์ƒ‰ ํ’ˆ์งˆ์„ ํ–ฅ์ƒ์‹œํ‚ค๋Š” ๋ฐ ์ค‘์š”ํ•œ ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค.


โœ… 1. explain ํŒŒ๋ผ๋ฏธํ„ฐ๋ž€?

๊ฒ€์ƒ‰ API ํ˜ธ์ถœ ์‹œ ์ฟผ๋ฆฌ ํŒŒ๋ผ๋ฏธํ„ฐ์— ?explain=true๋ฅผ ์ถ”๊ฐ€ํ•˜๋ฉด,
๊ฐ ๋ฌธ์„œ์˜ _score๊ฐ€ ์–ด๋–ป๊ฒŒ ๊ณ„์‚ฐ๋˜์—ˆ๋Š”์ง€ ์ž์„ธํ•œ ์„ค๋ช…์ด ํ•จ๊ป˜ ๋ฐ˜ํ™˜๋ฉ๋‹ˆ๋‹ค.

์˜ˆ์‹œ ํ˜ธ์ถœ

GET /posts/_search?explain=true
{
  "query": {
    "match": {
      "title": "์นด์นด์˜ค"
    }
  }
}

๐Ÿ“Š 2. BM25 ์ ์ˆ˜ ๊ณ„์‚ฐ ์š”์†Œ

BM25๋Š” ๊ธฐ๋ณธ์ ์œผ๋กœ ๋‹ค์Œ 3๊ฐ€์ง€ ์š”์†Œ๋กœ ์ ์ˆ˜๋ฅผ ๊ณ„์‚ฐํ•ฉ๋‹ˆ๋‹ค:

์š”์†Œ์„ค๋ช…
TF (Term Frequency)๋ฌธ์„œ ๋‚ด ๊ฒ€์ƒ‰์–ด ๋“ฑ์žฅ ํšŸ์ˆ˜
IDF (Inverse Document Frequency)์ „์ฒด ๋ฌธ์„œ ์ค‘ ํ•ด๋‹น ๊ฒ€์ƒ‰์–ด๊ฐ€ ๋“œ๋ฌธ ์ •๋„
fieldNormํ•„๋“œ ๊ธธ์ด์— ๋”ฐ๋ฅธ ์ •๊ทœํ™”

๐Ÿ” 3. explain ์ถœ๋ ฅ ์˜ˆ์‹œ

"_explanation": {
  "value": 1.472,
  "description": "weight(title:์นด์นด์˜ค in 0) [PerFieldSimilarity], result of:",
  "details": [
    {
      "value": 0.87,
      "description": "score(freq=2.0), computed as boost * idf * tf from:",
      "details": [
        { "value": 1.0, "description": "boost" },
        { "value": 0.92, "description": "idf, computed from docFreq" },
        { "value": 0.95, "description": "tf, term frequency" }
      ]
    },
    {
      "value": 1.23,
      "description": "fieldNorm based on field length"
    }
  ]
}
  • boost: ์ฟผ๋ฆฌ์—์„œ ์„ค์ •ํ•œ ๊ฐ€์ค‘์น˜
  • idf: ์ „์ฒด ๋ฌธ์„œ ์ค‘ ๋“œ๋ฌธ ๋‹จ์–ด์ผ์ˆ˜๋ก ๋†’์Œ
  • tf: ํ•ด๋‹น ๋ฌธ์„œ์—์„œ ์–ผ๋งˆ๋‚˜ ์ž์ฃผ ๋“ฑ์žฅํ•˜๋Š”์ง€
  • fieldNorm: ๋ฌธ์„œ ๊ธธ์ด ๋ณด์ •๊ฐ’

๐Ÿ› ๏ธ 4. ์ ์ˆ˜ ํŠœ๋‹ ์‹œ ํ™œ์šฉ ์˜ˆ

  • ํŠน์ • ํ•„๋“œ๊ฐ€ ๋„ˆ๋ฌด ๋งŽ์€ ํ…์ŠคํŠธ๋ฅผ ํฌํ•จํ•˜๋ฉด fieldNorm์ด ๋‚ฎ๊ฒŒ ๋‚˜์˜ด โ†’ ์ ์ˆ˜ ๋‚ฎ์Œ
  • idf๊ฐ€ ๋‚ฎ์œผ๋ฉด ํ”ํ•œ ๋‹จ์–ด โ†’ ๋‹ค๋ฅธ ๋‹จ์–ด์™€ ์กฐํ•ฉํ•˜๊ฑฐ๋‚˜ boost๋กœ ๊ฐ€์ค‘์น˜ ์กฐ์ • ํ•„์š”
  • explain์œผ๋กœ ๊ฐ ์š”์†Œ ๊ฐ’์„ ๋ณด๋ฉฐ ์›์ธ ์ง„๋‹จ ๊ฐ€๋Šฅ

๐Ÿงช 5. ์‹ค์ „ ํŒ

  • ?explain=true๋Š” ๋””๋ฒ„๊น… ์šฉ๋„๋กœ๋งŒ ์‚ฌ์šฉ (์„ฑ๋Šฅ ์ €ํ•˜ ์žˆ์Œ)
  • ์‹ค ์„œ๋น„์Šค์—์„œ๋Š” ์‚ฌ์šฉํ•˜์ง€ ์•Š๊ณ , ๋ถ„์„์šฉ์œผ๋กœ๋งŒ ํ˜ธ์ถœ
  • Kibana ๋˜๋Š” Dev Tools์—์„œ ์ง์ ‘ ํ…Œ์ŠคํŠธ ๊ฐ€๋Šฅ

โœ… ์š”์•ฝ

ํ•ญ๋ชฉ์„ค๋ช…
?explain=true๊ฒ€์ƒ‰ ๊ฒฐ๊ณผ์— _score ๊ณ„์‚ฐ ๊ทผ๊ฑฐ ์ถ”๊ฐ€
TF๋ฌธ์„œ ๋‚ด ๋‹จ์–ด ๋นˆ๋„
IDF๋‹จ์–ด์˜ ํฌ๊ท€๋„ (์ „์ฒด ๋ฌธ์„œ ๊ธฐ์ค€)
fieldNormํ•„๋“œ ๊ธธ์ด ๋ณด์ •๊ฐ’ (์งง์„์ˆ˜๋ก ๊ฐ€์ค‘์น˜ ๋†’์Œ)
ํ™œ์šฉ ๋ชฉ์ ๋†’์€ ์ ์ˆ˜์˜ ์ด์œ  ๋ถ„์„, ์ฟผ๋ฆฌ ๊ฐœ์„  ๋ฐ ํŠœ๋‹