Pointwise Mutual Information(PMI) compares the probability of two events occurring together to what this probability would be if the events were independent.

<aside> <img src="/icons/checkmark_green.svg" alt="/icons/checkmark_green.svg" width="40px" /> If two events and are independent, their PMI will be zero. A positive PMI indicates that and are more likely to co-occur than they would be if they were independent. Similarly, a negative PMI indicates that and are less likely to co-occur.

</aside>

<aside> <img src="/icons/checkmark_green.svg" alt="/icons/checkmark_green.svg" width="40px" /> Why?

• If two events are independent, PMI equals zero?

  Since event $w$ and event $c$ are independent,

  $$ P(w,c) = P(w) \cdot P(c) $$

  thus,

  $$ ⁍ $$

• Two events are more likely occur than they would be if they were independent, then PMI is positive?

  Since event $w$ and event $c$ are more likely to occur than they would be if they were independent,

  $$ P(w,c) > P(w) \cdot P(c) $$

  thus,

  $$ ⁍ $$

• Two events are less likely occur than they would be if they were independent, then PMI is negative?

  Since event $w$ and event $c$ are less likely to occur than they would be if they were independent,

  $$ P(w,c) < P(w) \cdot P(c)

  $$

  thus,

  $$ log\frac{P(w,c)}{P(w)P(c)} < log\frac{P(w)P(c)}{P(w)P(c)} = log(1) =0 $$

</aside>