The outer breakwater were the floating 'pontoons'. Some of the torpedoes would have detonated on those. Those were cabled together in a chain, and each with multiple anchors. The next breakwater were the hulks grounded bow to stern. Not much point in a torpedo hitting those. The inner breakwater were large concrete cassons, water filled and grounded. The piers had heads that were set on multiple columns, the pier connection to the beach were pontoons, linked & with multiple anchors.
The Mulberrys were in relatively shallow water. Shallower at low tide. They were also constructed inside the German established mine field. I dont know how long it took to clear that, tho its presence did suggest how far out from the coast the water remained shallow. There were also a lot of complex currents in the Channel, that vary by time of day.
The A Mulbeery was hit by a storm it was not designed for, had the triple breakwaters damaged to the point their efficiency was compromised 20% to 30%, had two of its three piers totally destroyed. It still took in cargo discharge in excess of its designed rate for near two months longer than intended. So yes torpedoes could blow things and damage parts. How many torpedoes would it take to equal the June storm?