Hey @alexeyrodriguez! 👋 I'm here to help with bugs, questions, and contributions. Let's squash those bugs together!

The issue you've identified is indeed a valid concern. The Ideal Discounted Cumulative Gain (IDCG) should be calculated based on the expected_ids, which represent the ideal order of relevant documents, rather than the retrieved_ids. This ensures that the IDCG reflects the best possible ordering of relevant documents.

To correct this, you should modify the calculation of idcg to consider the expected_ids. Here's a suggested change:

idcg = sum(
    discounted_gain(rel=True, i=i, mode=mode)
    for i in range(1, len(expected_ids) + 1)
)

This adjustment will ensure that the IDCG is calculated based on the ideal ranking of the expected relevant documents, aligning with the standard definition of NDCG ^[1].

To continue talking to Dosu, mention @dosu.