This has been working great for me for a long while - I started out doing this in Outlook, and then distilled it into a simpler, more effective approach for Mail.app:

http://the.taoofmac.com/space/blog/2013/02/24/2250

Summarizing:

- My "Inbox" is actually a search/smart folder
- I have an "Actionable" smart folder for stuff that needs doing/tracking/replying to
- I have a "Current" smart folder for easily filing away entire conversation threads
- MLs go straight to folders

I also have a "Mentions" smart folder that keeps track of e-mails where I'm mentioned in any way, but that's mostly for completion. Turns out I don't need to use it much.

Oh, and a "Week" smart folder that makes it easy to find anything sent or received over the last 7 days.

In practice, most stuff that is "actively ongoing" is in either Actionable or Current, and once it's dealt with it's filed away (one folder per quarter).

Junk or noise of any kind are, of course, ruthlessly deleted, but most of it never reaches my inbox.