There are probably a million and one ways of putting together a music collection, any of which will work. They all pretty much have the same thing in common: buy a piece of music and add it to your collection. So what do I have to offer? A couple of suggestions if you will...
1) Decide if you will collect CD's or MP3's or both. Personally I go for the CD's and from time to time add in some MP3's and convert a lot of my CD's into MP3's.
2) Determine a buying strategy. Mine is to purchase at least 2-3 albums every other month. In reality this works out to more like 5-7 CD's every 3-4 months, but at least it's consistent.
3) Spread out your purchases amongst the composers, genres, etc. of the classical music you do like. I tend to pick something up from Mahler or Dvorak, a Cello work, and then a couple of others amongst the various composers I like. And then every third or fourth purchase I pick up stuff that has pickled my interest. For example, I've come to really like what I've heard from Leos Janacek, but don't have anything of his yet. Guess what's on my shopping list?
4) If you use a site like
Arkiv Music, make use of it's wish list to keep a list of music that you are interested in but haven't gotten yet. I tend to pick up most everything but maybe one selection per purchase from my wish list and I keep around 100-120 items there. This way I am sure to be in the mood for something(s) on the list when I make my purchases.
Anyway hope this helps!