A Wall Street Journal article suggests that Romney's use of offshore entities in the Caymans, permits him to avoid the unrelated business income tax (UBIT), which can kick in when tax-exempt entities (such as pension funds) have debt-financed portfolio income.
This raises the question of whether we should view Romney (a) as having been engaged in disreputable tax avoidance behavior, demonstrating how the rich and well-advised can avoid intended income tax liability - or instead (b) as merely avoiding traps for the unwary and/or structuring his investments rationally and in a tax-efficient manner, as any well-advised investor would.
The question has no clear answer. Those who would like to plunge into the thickets a bit should definitely consult the article on using overseas entities that David Miller presented at the NYU Tax Policy Colloquium last year, which you can find here. At pages 40 through 48 of the article (following the numbers on the bottom of each page, rather than the overall document numbering at the link), he explains in detail both (a) the underlying UBIT rules and how they might have created a tax liability but for the apparently standard strategy that I gather Romney followed (investing through Caymans "blocker entities"), and (b) how the strategy eliminates the UBIT liability (which is pretty simple - the US tax-exempt doesn't itself borrow, but just gets dividends from its Caymans creature which does the borrowing).
The paper also tackles the further question of whether we should think there is anything wrong with allowing the strategy to work. The suggested conclusion (see pages 47-48) is that (a) Congress really did intend to apply the UBIT in situations where Romney's tax planning strategy permits him to avoid it, but (b) the provision that he is avoiding "was crafted less by prudent tax policy and more by politics," and hence one might not object strongly to its being avoidable.
UPDATE: I discuss the UBIT tax planning angle in a Christian Science Monitor article by Ron Scherer, available here.