To say your theory is far-fetched (what conspiracy isn't?
would be an understatement.
If anyone gains anything by driving away users from Kazaa, rest assured, it isn't Kazaa. Not now, and definitely not for the future. What good is a paid 'media desktop' when you don't have any users? What good is an 'advertising partner' when there are no people on the network to watch the ads? Paid features will be slipped in quietly into the existing product. Sure, I bet a sleek all-paid version is in the works, would be pretty short-sighted otherwise.
MPAA do what RIAA did before them; send spiders on the network to search for copyrighted material, track down the IP address and the ISP that owns it, and send a form letter. They hardly need Kazaa's cooperation for this.
As to knowing what's on their own network... well, they can find out the same way anyone else can, by quering every node for its file list. When you ask for recommendations, the engine looks at your list of files, or your list of recent downloads. It only makes sense, how else do you expect this to work? By magic? Whenever you have a third party 'recommend' something to you, you have to give up some information. No conspiracy there. Otherwise you'll get a list of random selections. (OTOH, that would probably work better
Whether the engine runs locally (downloads the logic to your machine) or by sending a list somewhere for processing is the only sticking point.
BTW, your ISP knows what you're downloading as well, because all Gnutella traffic is unencrypted. Just hide under a rock, it's probably as good a solution as any.