If Apollo knocked down the wall, the Boar would have entered the village. The wall was their only protection.
Didn't they have an army?
They never needed any protection except for the wall.
Let us know what you're thinking about Apollo, the people of the Town of Bliss, the Boar or anything else concerning Apollo the Misguided Missile or bullying.