When building a lightweight rifle, the most expensive place to conserve weight is the action.
In order of ease to conserve weight are:
1. Barrel length and taper
2. Blind magazine as opposed to bottom metal
3. Stock material
4. Action weight
And if you want to get crazy, caliber selection. For equal length and taper a .308 will be about .3 ounces lighter than a 6.5