Birthright: Chimaeron

I cannot direct hits like that, it is semi-random. I will see what happens if I put castle to move 0

Okay, move 0 it is. They just don't advance, which makes way more sense anyway. They fire from their square, and if people get into melee with them, they go to melee mode.

The rule was that a castle with no units inside counts as being defended by a single unit of irregulars, representing the garrison (which is dozens of soldiers already) and its built-in defenses (boiling oil and so forth).

As far as I know, you have to take the fortifications down to 0 before you can fully occupy a province, though you can just use units to neutralize it and occupy what isn't protected by the castle. I suppose a battle to capture the fortifications intact might be a way to avoid having to ruin them completely, though then you'd lose units instead of castle levels. I don't think that's the way we've ever done it, though.

Units can retreat inside a castle to avoid being destroyed when they rout during a province battle with nowhere else to retreat to. This is a pretty big benefit already.

No, you have to assault or siege it down to 0 before you can occupy the province. You can pillage out anything above castle level, though, without taking the castle.