a) $$5\sqrt{75} + \sqrt{2}(\sqrt{8} - \sqrt{24}) = 5\sqrt{25 \cdot 3} + \sqrt{2}(\sqrt{4 \cdot 2} - \sqrt{4 \cdot 6}) = 5 \cdot 5\sqrt{3} + \sqrt{2}(2\sqrt{2} - 2\sqrt{6}) = 25\sqrt{3} + 2\sqrt{2}\sqrt{2} - 2\sqrt{2}\sqrt{6} = 25\sqrt{3} + 2 \cdot 2 - 2\sqrt{12} = 25\sqrt{3} + 4 - 2\sqrt{4 \cdot 3} = 25\sqrt{3} + 4 - 2 \cdot 2\sqrt{3} = 25\sqrt{3} + 4 - 4\sqrt{3} = 4 + (25-4)\sqrt{3} = 4 + 21\sqrt{3}$$
Ответ: $$4 + 21\sqrt{3}$$
б) $$(\sqrt{8} - \sqrt{5})^{2} = (\sqrt{8})^{2} - 2\sqrt{8}\sqrt{5} + (\sqrt{5})^{2} = 8 - 2\sqrt{40} + 5 = 13 - 2\sqrt{4 \cdot 10} = 13 - 2 \cdot 2 \sqrt{10} = 13 - 4\sqrt{10}$$
Ответ: $$13 - 4\sqrt{10}$$