Simple references:
https://www.deeplearningbook.org/
https://www.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020
DON'T make the rookie mistake of going through all, or even two of them, at the same time. Pick one and stick to it. It'll be boring but rewarding.
Resources which make me happy 
https://www.amazon.com/Understanding-Machine-Learning-Theory-Algorithms/dp/1107057132/
https://www.amazon.com/dp/0262039400/
(note: the last one doesn't cover neural networks, but it does explain various loss functions well).