It all depends on the agreement that the family makes with the auctioneer. If they want to bid, that's fine, but if they bid the prices up past the point that the item doesn't sell, they still owe the auctioneer his commission.
It also depends on the various laws of that particular state. Some states have different types of auctions, (reserve or absolute), some states let the house bid, some don't.
