No one is disputing the fact that the best list is a huge list of buyers, people who've already bought from you and trust you and might buy from mailing list you for many years to come. But what of people merely enquiring about your product right now, people who might become customers once certain hurdles are overcome which currently prevent them from buying?
What if, for example, those people are short of cash and unable to afford your product until their next pay day; what if they're at work when they read your sales letter and they prefer to order later from the privacy of home? Or maybe your product is close to what your contact is desperate to buy but not quite close enough. That person may want a different colour, for example, or a different size, or perhaps he or she doesn't trust you yet and prefers to learn more about you before giving you their hard-earned cash.
Whatever the reason people choose not to buy right away, it's safe to say if you don't take their contact details you'll never hear from mailing list them again. And because a large proportion of those people might otherwise become repeat, high ticket buyers, it's a good idea to get them onto your mailing list, alongside buyers, to tempt a purchase some time soon.