Sure...
The way I would design it (all REAL WOOD and heavy timber) would be a 20' x 20' free standing structural frame (BIG TIMBER).
Walls would be non load bearing... inside features such as lofts, bathrooms, kitchen etc. would be like "adding parts"... and 100% independent from the structure.