.22 is more effective although .177 will get the job done given enough velocity and good shot placement. Good shot placement is required with any pellet gun though. No .22 or .177 caliber pellet rifle has anywhere near the killing power of a .22 LR HP. For your specific need I suggest a multi pump pneumatic. Pump it just 2 or 3 times for routing the neighbors dogs. Otherwise you'll be burying pellets deep in their ass.
I give up. What does killing rabbits and squirrels have to do with shooting dogs. Original poster never mentioned the neighbor’s dogs or a need to shoot them. Geez, guy, read a little more carefully....
