You’ve probably solved this by now, but for the sake of others who hope to make Sketch their Adobe killer (like me) here’s the real answer.

Sketch’s arrow shape tool sucks.

You’re problem lies in the stroke end style. Take a look at these arrows and their corresponding stroke end settings.

1. This one is sucky.
   

2. This one is less so.
   

I’ve also noticed a serious screen drawing issue with the arrow shape but not with other shape objects. I think there’s just a bug with the poor thing.

Sketch is a young app. These things are bound to plague us hopeful early users. Fortunately, Bohemian just brought on two new employees to keep things moving ahead. Let’s all hope for something great in Sketch 3.0!