Not sure where you live, but if there's any chance of visiting a store that sells modular, I couldn't recommend that approach enough, even if it means planning a vacation around it. I was turned on to modular serendipitously by a visit to Rough Trade East in London which had a SchneidersLaden demo room in the back. There's absolutely nothing to compare to hands-on experience with a little in-person expertise to help guide you.
If you do the second hand experience - be very careful. Incredibly, even for expensive modules people often just produce non-descriptive lists of gear without photo or condition. You'll want something in good cosmetic condition - if you love the module, you'll be looking at it for a long time, if you re-sell it, you don't want any difficulties making the next buyer unhappy with it's appearance. You may also want to shoot video of your rig, so it's not enough that the module is simply functional.
Take time to do your research, watch a lot of videos and don't buy too much all at once. Check also the service reputations of the makers - hopefully you won't need it, but it's sure frustrating when after buying a buggy module for $500 you discover that the maker isn't returning anyone's email.
NEW EP: https://fastus.bandcamp.com/album/terra-incognito
Latest works: https://soundcloud.com/fastus
Modular studies & sketches: https://soundcloud.com/problivion