We don't have that problem with our '19 Highlander, I agree it seems like a phone issue. The only time my phone (or my wife's phone) connects to the bluetooth is when the car is on. When you turn it on our phones connect automatically and when we shut it off they automatically disconnect.
