Having carried a Kel-Tec P32 (.32 Auto) in my front jeans pocket for a few years, I'd recommend getting either the Kel-Tec P3AT or Ruger LCP, both .380's.
Reasons are:
Size difference is negligible, where the P32 disappears into a pocket so will either .380.
.380 gives a little more oomph.
Disregarding the current ammo shortages, even in "normal" times .32 ammo is more scarce than .380 and is much more expensive due to the lesser demand.
My current pocket pistol is a Kahr CM9 - Concealed Micro 9 - and it technically does fit into my pocket but it is definitely larger than either of the .380's mentioned.
