As far as budget goes there are many bows used and new under $400 set up to get you started. Work to be proficient at 40 yards. I would recommend a 50-60 pound bow, especially if you have not shot bows alot as it will make practice easier and build confidence faster. If you already hunt you probably have most of the other equipment you need. Only difference is early season is warmer and drier than rifle usually. Land closures can be a problem. As far as drawbacks, other than range, I can't think of any.
