If you can see them approaching, don't you try to immediately make space for them?
You probably do, know I do. Some people do make it hard, but when I see merging traffic (as I'm usually in the right lane unless passing), I start trying to make a space for them.