Buildings and Defensive Towers are featured structures of Clash of Beasts and there is no way to destroy them. We may add a feature where you can sell your towers on future updates.
You can store Defensive Towers in your inventory and use them as you will.