What is the difference between minimizing a function f mathematicaly, and minimizing f for deep learning purposes?

A.:

Minimizing f mathematically means finding the smallest value of f in its range (global minimum). Minimizing f in the context of deep learning usually means decreasing f a lot, not necessarily reaching a minimum, either local or global.