Per the question...you have to know what silver or any other pm you're dealing for looks like, what the markings are and what they indicate and know the signs of tampering. Probably the best go is to get pre-65 silver US coinage, which is what everybody knows. Easiest to determine that it's real whether you're buying or selling. Morgan and Peace Dollars are okay. Big hunks of silver with pretty designs and common years not worth a big premium in numismatic (collector's) value. Better are the more common denominations though, as everybody knows them. Walkers are cool, but everybody knows Kennedy halves. Mercs are cool but everybody knows Roosevelts. Common junk silver.

The thing about silver and gold is not wealth accumulation but wealth preservation. If you buy in order to make "money" you may or may not be happy with the results, just like the stock market. You're also tying yourself to the same system you supposedly disdain since in order to build more "wealth" you have to sell your PM's in order to get paper. If you buy PM's to preserve your wealth, that's a different story. Paper may be worth a lot today and virtually nothing tomorrow. PM's will always be worth something.