First, it adds a parameter to limit the upper boundary of TF value. Second, it introduces average document length to measure the influence.

Why BM-25 is slightly Better than TF-IDF?

The BM25 formula is a document scoring method in information retrieval that improves upon TF-IDF. It introduces parameters to limit the upper boundary of the term frequency (TF) value and uses average document length to measure influence. The score for a query $$q$$ and document $$d$$ is defined as:

$$\sum_{t \in q} \frac{N}{df_t} \cdot \frac{tf_{t,d}}{k(1-b+ b \frac{|d|}{|d_{avg}|}) + tf_{t,d}}$$

University of Michigan - Ann Arbor

Google

The tf-idf algorithm is a method for weighting co-occurrence matrices in information retrieval, usually used when the dimensions are documents. The algorithm is the product of two terms: - Term frequency is the frequency of the word $$t$$ in the document $$d$$, commonly denoted in log weighting as $$tf_{t, d} = log_{10}(count(t, d) + 1)$$. - Inverse document frequency is also usually denoted in a log function as $$idf_t = log_{10}\left(\frac{N}{df_t}\right)$$, where document frequency $$df_t$$ of a term $$t$$ is the number of documents in which it occurs and $$N$$ is the total number of documents in the collection. The tf-idf weighted value $$w_{t, d}$$ for word $$t$$ in document $$d$$ is thus defined as $$w_{t, d} = tf_{t, d} \times idf_t$$

TF-IDF Algorithm

Document is scored according to the similarity with query to decide the rank of it in the response list. The plain score can be expressed as:
$score(q,d) = cos(q,d) = \frac{\textbf(q)}{|\textbf(q)|} \cdot \frac{\textbf(d)}{|\textbf(d)|} $

Document Scoring

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

The tf-idf model can be used to compute word similarity by computing the cosine of two word vectors

The tf-idf vector models can also be used to decide if two documents are similar. 
For each document that we want to compare,  we need to compute the centroid for all their vectors. 
Given $$k$$ word vectors $$w_1,w_2,....,w_k$$, the centroid document vector $$d$$ is:

$$
d = \frac{{w_1+w_2+...+w_k}}{k}
$$

Then we can compute the $$cos(d_1,d_2)$$ to estimate the similarity of these two documents: high cosine, high similarity.

Application of the tf-idf vector models

 $idf_t = log_{10} \frac{N}{df_t}$ where N is the total number of documents in the collection.

Inverse Document Frequency(IDF)

Term Frequency(TF)

BM25 Document Scoring Formula

The TF-IDF version of the document scoring formula computes the cosine similarity between the TF-IDF vector of a query $$q$$ and the TF-IDF vector of a document $$d$$:

$$score(q,d) = \sum_{t \in q} \frac{tf-idf(t,q)}{\sqrt{\sum_{q_i \in q} tf-idf^2(q_i,q)}} \cdot \frac{tf-idf(t,d)}{\sqrt{\sum_{d_i \in d} tf-idf^2(d_i,d)}}$$

Where:
- $$t$$ is a term present in the query $$q$$ and the document $$d$$.
- $$tf-idf(t,q)$$ is the TF-IDF weight of term $$t$$ in query $$q$$.
- $$tf-idf(t,d)$$ is the TF-IDF weight of term $$t$$ in document $$d$$.
- The denominators represent the Euclidean ($$L_2$$) norms of the query and document vectors, respectively, which normalize the score to be independent of document and query lengths.

Learn Before

Related

Learn After