In a canonical model of heterogeneous agents with precautionary saving motives, Aiyagari (1995) breaks the classical result of zero capital tax obtained in representative-agent models. Aiyagari argues that with capital overaccumulation the optimal long-run capital tax should be strictly positive in order to achieve aggregate allocative efficiency suggested by the modified golden rule (MGR). In this paper, we find that, depending on the sources of capital overaccumulation, capital taxation may not be the most efficient means to restore the MGR when government debt is feasible. To demonstrate our point, we study optimal policy mix in achieving the socially optimal (MGR) level of aggregate capital stock in an infinite horizon heterogeneous-agents incomplete-markets economy where capital may be overaccumulated for two distinct reasons: (i) precautionary savings and (ii) production externalities. By solving the Ramsey problem analytically along the entire transitional path, we reveal that public debt and capital taxation play very distinct roles in dealing with the overaccumulation problem. The Ramsey planner opts neither to use a capital tax to correct the overaccumulation problem if it is caused solely by precautionary saving—regardless of the feasibility of public debt—nor to use debt (financed by consumption tax) to correct the overaccumulation problem if it is caused solely by production externality (such as pollution)—regardless of the feasibility of a capital tax. The key is that the MGR has two margins: an intratemporal margin pertaining to the marginal product of capital (MPK) and an intertemporal margin pertaining to the time discount rate. To achieve the MGR, the Ramsey planner needs to equate not only the private MPK with the social MPK but also the interest rate with the time discount rate—neither of which is equalized in a competitive equilibrium. Yet public debt and a capital tax are each effective only in calibrating one of the two margins, respectively, but not both.