If you’re not interested in purchasing an infuser mug or using paper filters, there’s no reason you can’t brew tea at home in your own mugs. All you need is an infuser and those can be obtained at a variety of price points.
The easiest, cheapest alternative is to purchase a tea ball. You can get these at any grocery store, or if you’re already ordering from Amazon this one is available for $2.99:
If you’re looking for more nerd cred, I suggest checking out the infusers at ThinkGeek.
Or if you’re looking for a bit more whimsy, try one of these: