We analyze the Ramsey planner's decisions to finance stochastic public expenditures under incomplete insurance markets for idiosyncratic risk. We show analytically that whenever the market interest rate lies below the time discount rate, the Ramsey planner has a dominant incentive to increase debt to meet the private sector's demand for full self-insurance regardless of the relative size of aggregate shocks---suggesting a departure from tax smoothing. However, if a full self-insurance Ramsey allocation is infeasible in the absence of a government debt limit, an interior or bounded Ramsey equilibrium does not exist. The strong incentives for the Ramsey planner to smooth both individual consumption (via increasing public debt) and aggregate consumption (via tax smoothing) imply that (i) the long-run Ramsey equilibrium is characterized by full self-insurance and constant taxes if state-contingent bonds are available and (ii) when state-contingent bonds are not available, the government's attempt to balance the competing incentives between tax smoothing and individual consumption smoothing---even at the cost of extra tax distortion---implies a bounded stochastic unit root component in optimal taxes and in the bond supply. In all cases considered in this paper, a sufficiently high average level of public debt (financed by distortionary taxation) to support full self-insurance is desirable and welfare improving. Therefore, adding a liquidity premium into the value of government bonds via incomplete financial markets can bring the theory of public finance into closer conformity with realty.