Primer is not a bad idea. Regardless, I would still sand down with 320 to give some tooth for the cerakote to adhere to.
Here is my tikka t3 lite synthetic stock that I painted a few weeks ago before deer hunting. For prep I masked off everything that I didn't want painted (bedding, etc.) then I put her in the sandblaster and lightly blasted the stock. All the paint is cerakote.