Can you discuss it easily, work things out, reach agreements, and feel happy at the end of the discussion?
Welcome to the club. You have a LOT of company, so don't beat yourself up about it.
It doesn't have to be so difficult. There are easier, better ways.
Did you know that ultra-affluent families establish entire offices to help coordinate, communicate, and manage? How do you suppose they progressed from humble origins to established, full-fledged family offices?