tax_smoothing_2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='tax-smoothing-2'></a>\n",
    "<div id=\"qe-notebook-header\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>\n",
    "\n",
    "\n",
    "<a id='index-0'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Pay for a War: Part 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [How to Pay for a War: Part 2](#How-to-Pay-for-a-War:-Part-2)  \n",
    "  - [An Application of Markov Jump Linear Quadratic Dynamic Programming](#An-Application-of-Markov-Jump-Linear-Quadratic-Dynamic-Programming)  \n",
    "  - [Two example specifications](#Two-example-specifications)  \n",
    "  - [A Model with Two-period Debt and No Restructuring](#A-Model-with-Two-period-Debt-and-No-Restructuring)  \n",
    "  - [Mapping the Two-period Model into an LQ Markov Jump Problem](#Mapping-the-Two-period-Model-into-an-LQ-Markov-Jump-Problem)  \n",
    "  - [Example Showing the Importance of the Penalty on Different Issuance Across Maturities](#Example-Showing-the-Importance-of-the-Penalty-on-Different-Issuance-Across-Maturities)  \n",
    "  - [A Model with Restructuring](#A-Model-with-Restructuring)  \n",
    "  - [Model with Restructuring as a Markov Jump Linear Quadratic Control Problem](#Model-with-Restructuring-as-a-Markov-Jump-Linear-Quadratic-Control-Problem)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Co-author**: [Sebastian Graves](https://github.com/sebgraves)\n",
    "\n",
    "In addition to what&rsquo;s in Anaconda, this lecture will need the following libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": true
   },
   "outputs": [],
   "source": [
    "!pip install --upgrade quantecon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An Application of Markov Jump Linear Quadratic Dynamic Programming\n",
    "\n",
    "This is a [sequel to an earlier lecture](https://lectures.quantecon.org/tax_smoothing_1.html).\n",
    "\n",
    "We use a method introduced in lecture [Markov Jump LQ dynamic programming](https://lectures.quantecon.org/markov_jump_lq.html) to\n",
    "implement suggestions by Barro (1999 [[Bar99]](https://lectures.quantecon.org/zreferences.html#barro1999determinants), 2003 [[BM03]](https://lectures.quantecon.org/zreferences.html#barro2003religion)) for extending his\n",
    "classic 1979 model of tax smoothing.\n",
    "\n",
    "Barro&rsquo;s 1979 [[Bar79]](https://lectures.quantecon.org/zreferences.html#barro1979) model is about a government that borrows and lends in order\n",
    "to help it minimize an intertemporal measure of distortions caused by\n",
    "taxes.\n",
    "\n",
    "Technically, Barro&rsquo;s 1979 [[Bar79]](https://lectures.quantecon.org/zreferences.html#barro1979) model looks a lot like a consumption-smoothing model.\n",
    "\n",
    "Our generalizations of his 1979 [[Bar79]](https://lectures.quantecon.org/zreferences.html#barro1979) model will also look\n",
    "like souped-up consumption-smoothing models.\n",
    "\n",
    "Wanting tractability induced Barro in 1979 [[Bar79]](https://lectures.quantecon.org/zreferences.html#barro1979) to assume that\n",
    "\n",
    "- the government trades only one-period risk-free debt, and  \n",
    "- the one-period risk-free interest rate is constant  \n",
    "\n",
    "\n",
    "In our [earlier lecture](https://lectures.quantecon.org/tax_smoothing_1.html), we relaxed the second of these assumptions but not the first.\n",
    "\n",
    "In particular, we used *Markov jump linear quadratic dynamic programming*\n",
    "to allow the exogenous interest rate to vary over time.\n",
    "\n",
    "In this lecture, we add a maturity composition decision to the\n",
    "government&rsquo;s problem by expanding the dimension of the state.\n",
    "\n",
    "We assume\n",
    "\n",
    "- that the government borrows or saves in the form of risk-free bonds\n",
    "  of maturities $ 1, 2, \\ldots , H $.  \n",
    "- that interest rates on those bonds are time-varying and in particular are\n",
    "  governed by a jointly stationary stochastic process.  \n",
    "\n",
    "\n",
    "Let&rsquo;s start with some standard imports:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import quantecon as qe\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two example specifications\n",
    "\n",
    "We&rsquo;ll describe two possible specifications\n",
    "\n",
    "- In one, each period the government issues zero-coupon\n",
    "  bonds of one- and two-period maturities and redeems them only when\n",
    "  they mature &ndash; in this version, the maturity structure of government\n",
    "  debt at each date is partly inherited from the past.  \n",
    "- In the second, the government redesigns the maturity\n",
    "  structure of the debt each period.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Model with Two-period Debt and No Restructuring\n",
    "\n",
    "Let $ T_t $ denote tax collections, $ \\beta $ a discount factor,\n",
    "$ b_{t,t+1} $ time $ t+1 $ goods that the government promises to\n",
    "pay at $ t $, $ b_{t,t+2} $ time $ t+2 $ goods that the\n",
    "government promises to pay at time $ t $, $ G_t $ government\n",
    "purchases, $ p_{t,t+1} $ the number of time $ t $ goods received\n",
    "per time $ t+1 $ goods promised, and $ p_{t,t+2} $ the number of\n",
    "time $ t $ goods received per time $ t+2 $ goods promised.\n",
    "\n",
    "Evidently, $ p_{t, t+1}, p_{t,t+2} $ are inversely related to\n",
    "appropriate corresponding gross interest rates on government debt.\n",
    "\n",
    "In the spirit of Barro (1979) [[Bar79]](https://lectures.quantecon.org/zreferences.html#barro1979), government\n",
    "expenditures are governed by an exogenous stochastic process.\n",
    "\n",
    "Given initial conditions $ b_{-2,0}, b_{-1,0}, z_0, i_0 $, where\n",
    "$ i_0 $ is the initial Markov state, the government chooses a\n",
    "contingency plan for $ \\{b_{t, t+1}, b_{t,t+2}, T_t\\}_{t=0}^\\infty $\n",
    "to maximize.\n",
    "\n",
    "$$\n",
    "- E_0 \\sum_{t=0}^\\infty \\beta^t \\left[ T_t^2 + c_1( b_{t,t+1} - b_{t,t+2})^2 \\right]\n",
    "$$\n",
    "\n",
    "subject to the constraints\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "T_t & = G_t + b_{t-2,t} + b_{t-1,t} - p_{t,t+2} b_{t,t+2} - p_{t,t+1} b_{t,t+1} \\cr\n",
    "  G_t &  = U_{g,t} z_t \\cr\n",
    "  z_{t+1} & = A_{22,t} z_t + C_{2,t} w_{t+1} \\cr\n",
    "  \\begin{bmatrix}\n",
    "   p_{t,t+1} \\cr\n",
    "   p_{t,t+2} \\cr\n",
    "   U_{g,t} \\cr\n",
    "   A_{22,t} \\cr\n",
    "   C_{2,t}\n",
    "  \\end{bmatrix} & \\sim \\textrm{functions of Markov state with transition matrix } \\Pi \\end{aligned}\n",
    "$$\n",
    "\n",
    "Here $ w_{t+1} \\sim {\\cal N}(0,I) $ and $ \\Pi_{ij} $ is\n",
    "the probability that the Markov state moves from state $ i $ to\n",
    "state $ j $ in one period.\n",
    "\n",
    "The variables\n",
    "$ T_t, b_{t, t+1}, b_{t,t+2} $ are *control* variables chosen at\n",
    "$ t $, while the variables $ b_{t-1,t}, b_{t-2,t} $ are\n",
    "endogenous state variables inherited from the past at time $ t $ and\n",
    "$ p_{t,t+1}, p_{t,t+2} $ are exogenous state variables at time\n",
    "$ t $.\n",
    "\n",
    "The parameter $ c_1 $ imposes a penalty on the government&rsquo;s issuing\n",
    "different quantities of one and two-period debt.\n",
    "\n",
    "This penalty deters the\n",
    "government from taking large &ldquo;long-short&rdquo; positions in debt of different\n",
    "maturities. An example below will show this in action.\n",
    "\n",
    "As well as extending the model to allow for a maturity decision for\n",
    "government debt, we can also in principle allow the matrices\n",
    "$ U_{g,t}, A_{22,t}, C_{2,t} $ to depend on the Markov state."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mapping the Two-period Model into an LQ Markov Jump Problem\n",
    "\n",
    "First, define\n",
    "\n",
    "$$\n",
    "\\hat b_t = b_{t-1,t} + b_{t-2,t} ,\n",
    "$$\n",
    "\n",
    "which is debt due at time $ t $.\n",
    "\n",
    "Then define the endogenous part of the state:\n",
    "\n",
    "$$\n",
    "\\bar b_t = \\begin{bmatrix}\n",
    " \\hat b_t \\cr\n",
    "   b_{t-1,t+1}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and the complete state\n",
    "\n",
    "$$\n",
    "x_t = \\begin{bmatrix} \\bar b_t \\cr\n",
    "        z_t\n",
    "      \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "and the control vector\n",
    "\n",
    "$$\n",
    "u_{t} =  \\begin{bmatrix}\n",
    "   b_{t,t+1} \\cr\n",
    "   b_{t,t+2}\n",
    "  \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The endogenous part of state vector follows the law of motion:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "    \\hat b_{t+1} \\cr\n",
    "    b_{t,t+2}\n",
    "   \\end{bmatrix}\n",
    "   =\n",
    "   \\begin{bmatrix}\n",
    "    0 & 1 \\cr\n",
    "    0 & 0\n",
    "   \\end{bmatrix}\n",
    " \\begin{bmatrix}\n",
    "  \\hat b_{t} \\cr\n",
    "  b_{t-1,t+1}\n",
    " \\end{bmatrix}\n",
    "+\n",
    "  \\begin{bmatrix}\n",
    "   1 & 0 \\cr\n",
    "   0 & 1 \\cr\n",
    "  \\end{bmatrix}\n",
    "   \\begin{bmatrix}\n",
    "    b_{t,t+1} \\cr\n",
    "    b_{t,t+2}\n",
    "   \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\n",
    "\\bar b_{t+1} = A_{11} \\bar b_t + B_1 u_t\n",
    "$$\n",
    "\n",
    "Define the following functions of the state\n",
    "\n",
    "$$\n",
    "G_t = S_{G,t} x_t, \\quad \\hat b_t = S_1 x_t\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "M_t  =  \\begin{bmatrix} - p_{t,t+1} & - p_{t,t+2} \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "where $ p_{t,t+1} $ is the discount on one period loans in the\n",
    "discrete Markov state at time $ t $ and $ p_{t,t+2} $ is the\n",
    "discount on two-period loans in the discrete Markov state.\n",
    "\n",
    "Define\n",
    "\n",
    "$$\n",
    "S_t = S_{G,t} + S_1\n",
    "$$\n",
    "\n",
    "Note that in discrete Markov state $ i $\n",
    "\n",
    "$$\n",
    "T_t = M_t u_t + S_t x_t\n",
    "$$\n",
    "\n",
    "It follows that\n",
    "\n",
    "$$\n",
    "T_t^2 = x_t' S_t' S_t x_t + u_t' M_t' M_t u_t + 2 u_t' M_t' S_t x_t\n",
    "$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\n",
    "T_t^2 = x_t'R_t x_t + u_t' Q_t u_t + 2 u_t' W_t x_t\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "R_t = S_t'S_t, \\quad Q_t = M_t' M_t, \\quad  W_t = M_t' S_t\n",
    "$$\n",
    "\n",
    "Because the payoff function also includes the penalty parameter on\n",
    "issuing debt of different maturities, we have:\n",
    "\n",
    "$$\n",
    "T_t^2 + c_1( b_{t,t+1} - b_{t,t+2})^2 = x_t'R_t x_t + u_t' Q_t u_t + 2 u_t' W_t x_t +  c_1 u_t'Q^c u_t\n",
    "$$\n",
    "\n",
    "where $ Q^c = \\begin{bmatrix} 1 & -1 \\\\ -1 & 1 \\end{bmatrix} $. Therefore, the overall $ Q $ matrix for the Markov jump LQ problem is:\n",
    "\n",
    "$$\n",
    "Q_t^c = Q_t + c_1Q^c\n",
    "$$\n",
    "\n",
    "The law of motion of the state in all discrete Markov states $ i $\n",
    "is\n",
    "\n",
    "$$\n",
    "x_{t+1} = A_t x_t +  B u_t +  C_t w_{t+1}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "A_t = \\begin{bmatrix}  A_{11} & 0 \\cr\n",
    "                        0 & A_{22,t}\n",
    "      \\end{bmatrix}, \\quad\n",
    "       B = \\begin{bmatrix}\n",
    "                  B_1 \\cr\n",
    "                  0\n",
    "                 \\end{bmatrix}, \\quad\n",
    "        C_t = \\begin{bmatrix} 0 \\cr C_{2,t} \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Thus, in this problem all the matrices apart from $ B $ may depend\n",
    "on the Markov state at time $ t $.\n",
    "\n",
    "As shown in the [previous lecture](https://lectures.quantecon.org/tax_smoothing_1.html),\n",
    "the `LQMarkov` class can solve Markov jump LQ problems when given the\n",
    "$ A, B, C, R, Q, W $ matrices for each Markov state.\n",
    "\n",
    "The function below maps the primitive matrices and parameters from the above\n",
    "two-period model into the matrices that the `LQMarkov` class requires:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "def LQ_markov_mapping(A22, C2, Ug, p1, p2, c1=0):\n",
    "\n",
    "    \"\"\"\n",
    "    Function which takes A22, C2, Ug, p_{t, t+1}, p_{t, t+2} and penalty\n",
    "    parameter c1, and returns the required matrices for the LQMarkov\n",
    "    model: A, B, C, R, Q, W.\n",
    "    This version uses the condensed version of the endogenous state.\n",
    "    \"\"\"\n",
    "\n",
    "    # Make sure all matrices can be treated as 2D arrays\n",
    "    A22 = np.atleast_2d(A22)\n",
    "    C2 = np.atleast_2d(C2)\n",
    "    Ug = np.atleast_2d(Ug)\n",
    "    p1 = np.atleast_2d(p1)\n",
    "    p2 = np.atleast_2d(p2)\n",
    "\n",
    "    # Find the number of states (z) and shocks (w)\n",
    "    nz, nw = C2.shape\n",
    "\n",
    "    # Create A11, B1, S1, S2, Sg, S matrices\n",
    "    A11 = np.zeros((2, 2))\n",
    "    A11[0, 1] = 1\n",
    "\n",
    "    B1 = np.eye(2)\n",
    "\n",
    "    S1 = np.hstack((np.eye(1), np.zeros((1, nz+1))))\n",
    "    Sg = np.hstack((np.zeros((1, 2)), Ug))\n",
    "    S = S1 + Sg\n",
    "\n",
    "    # Create M matrix\n",
    "    M = np.hstack((-p1, -p2))\n",
    "\n",
    "    # Create A, B, C matrices\n",
    "    A_T = np.hstack((A11, np.zeros((2, nz))))\n",
    "    A_B = np.hstack((np.zeros((nz, 2)), A22))\n",
    "    A = np.vstack((A_T, A_B))\n",
    "\n",
    "    B = np.vstack((B1, np.zeros((nz, 2))))\n",
    "\n",
    "    C = np.vstack((np.zeros((2, nw)), C2))\n",
    "\n",
    "    # Create Q^c matrix\n",
    "    Qc = np.array([[1, -1], [-1, 1]])\n",
    "\n",
    "    # Create R, Q, W matrices\n",
    "\n",
    "    R = S.T @ S\n",
    "    Q = M.T @ M + c1 * Qc\n",
    "    W = M.T @ S\n",
    "\n",
    "    return A, B, C, R, Q, W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the above function, we can proceed to solve the model in two steps:\n",
    "\n",
    "1. Use `LQ_markov_mapping` to map\n",
    "  $ U_{g,t}, A_{22,t}, C_{2,t}, p_{t,t+1}, p_{t,t+2} $ into the\n",
    "  $ A, B, C, R, Q, W $ matrices for each of the $ n $ Markov states.  \n",
    "1. Use the `LQMarkov` class to solve the resulting n-state Markov\n",
    "  jump LQ problem.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Showing the Importance of the Penalty on Different Issuance Across Maturities\n",
    "\n",
    "To implement a simple example of the two-period model, we assume that\n",
    "$ G_t $ follows an AR(1) process:\n",
    "\n",
    "$$\n",
    "G_{t+1} = \\bar G + \\rho G_t + \\sigma w_{t+1}\n",
    "$$\n",
    "\n",
    "To do this, we set $ z_t = \\begin{bmatrix} 1 \\\\ G_t \\end{bmatrix} $,\n",
    "and consequently:\n",
    "\n",
    "$$\n",
    "A_{22} = \\begin{bmatrix} 1 & 0 \\\\ \\bar G & \\rho \\end{bmatrix} \\hspace{2mm} , \\hspace{2mm} C_2 = \\begin{bmatrix} 0 \\\\ \\sigma \\end{bmatrix} \\hspace{2mm} , \\hspace{2mm} U_g = \\begin{bmatrix} 0 & 1 \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Therefore, in this example, $ A_{22}, C_2 $ and $ U_g $ are not\n",
    "time-varying.\n",
    "\n",
    "We will assume that there are two Markov states, one with a\n",
    "flatter yield curve, and one with a steeper yield curve. In state 1,\n",
    "prices are:\n",
    "\n",
    "$$\n",
    "p^1_{t,t+1} = \\beta \\hspace{2mm} , \\hspace{2mm} p^1_{t,t+2} = \\beta^2 - 0.02\n",
    "$$\n",
    "\n",
    "and in state 2, prices are:\n",
    "\n",
    "$$\n",
    "p^2_{t,t+1} = \\beta \\hspace{2mm} , \\hspace{2mm}  p^2_{t,t+2} = \\beta^2 + 0.02\n",
    "$$\n",
    "\n",
    "We first solve the model with no penalty parameter on different issuance\n",
    "across maturities, i.e. $ c_1 = 0 $.\n",
    "\n",
    "We also need to specify a\n",
    "transition matrix for the Markov state, we use:\n",
    "\n",
    "$$\n",
    "\\Pi = \\begin{bmatrix} 0.9 & 0.1 \\\\ 0.1 & 0.9 \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Thus, each Markov state is persistent, and there is an equal chance of\n",
    "moving from one to the other."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Model parameters\n",
    "&beta;, Gbar, &rho;, &sigma;, c1 = 0.95, 5, 0.8, 1, 0\n",
    "p1, p2, p3, p4 = &beta;, &beta;**2 - 0.02, &beta;, &beta;**2 + 0.02\n",
    "\n",
    "# Basic model matrices\n",
    "A22 = np.array([[1, 0], [Gbar, &rho;] ,])\n",
    "C_2 = np.array([[0], [&sigma;]])\n",
    "Ug = np.array([[0, 1]])\n",
    "\n",
    "A1, B1, C1, R1, Q1, W1 = LQ_markov_mapping(A22, C_2, Ug, p1, p2, c1)\n",
    "A2, B2, C2, R2, Q2, W2 = LQ_markov_mapping(A22, C_2, Ug, p3, p4, c1)\n",
    "\n",
    "# Small penalties on debt required to implement no-Ponzi scheme\n",
    "R1[0, 0] = R1[0, 0] + 1e-9\n",
    "R2[0, 0] = R2[0, 0] + 1e-9\n",
    "\n",
    "# Construct lists of matrices correspond to each state\n",
    "As = [A1, A2]\n",
    "Bs = [B1, B2]\n",
    "Cs = [C1, C2]\n",
    "Rs = [R1, R2]\n",
    "Qs = [Q1, Q2]\n",
    "Ws = [W1, W2]\n",
    "\n",
    "&Pi; = np.array([[0.9, 0.1],\n",
    "              [0.1, 0.9]])\n",
    "\n",
    "# Construct and solve the model using the LQMarkov class\n",
    "lqm = qe.LQMarkov(&Pi;, Qs, Rs, As, Bs, Cs=Cs, Ns=Ws, beta=&beta;)\n",
    "lqm.stationary_values()\n",
    "\n",
    "# Simulate the model\n",
    "x0 = np.array([[100, 50, 1, 10]])\n",
    "x, u, w, t = lqm.compute_sequence(x0, ts_length=300)\n",
    "\n",
    "# Plot of one and two-period debt issuance\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n",
    "ax1.plot(u[0, :])\n",
    "ax1.set_title('One-period debt issuance')\n",
    "ax1.set_xlabel('Time')\n",
    "ax2.plot(u[1, :])\n",
    "ax2.set_title('Two-period debt issuance')\n",
    "ax2.set_xlabel('Time')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above simulations show that when no penalty is imposed on different\n",
    "issuances across maturities, the government has an incentive to take\n",
    "large &ldquo;long-short&rdquo; positions in debt of different maturities.\n",
    "\n",
    "To prevent such an outcome, we now set $ c_1 = 0.01 $.\n",
    "\n",
    "This penalty is enough\n",
    "to ensure that the government issues positive quantities of both one and\n",
    "two-period debt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Put small penalty on different issuance across maturities\n",
    "c1 = 0.01\n",
    "\n",
    "A1, B1, C1, R1, Q1, W1 = LQ_markov_mapping(A22, C_2, Ug, p1, p2, c1)\n",
    "A2, B2, C2, R2, Q2, W2 = LQ_markov_mapping(A22, C_2, Ug, p3, p4, c1)\n",
    "\n",
    "# Small penalties on debt required to implement no-Ponzi scheme\n",
    "R1[0, 0] = R1[0, 0] + 1e-9\n",
    "R2[0, 0] = R2[0, 0] + 1e-9\n",
    "\n",
    "# Construct lists of matrices\n",
    "As = [A1, A2]\n",
    "Bs = [B1, B2]\n",
    "Cs = [C1, C2]\n",
    "Rs = [R1, R2]\n",
    "Qs = [Q1, Q2]\n",
    "Ws = [W1, W2]\n",
    "\n",
    "# Construct and solve the model using the LQMarkov class\n",
    "lqm2 = qe.LQMarkov(&Pi;, Qs, Rs, As, Bs, Cs=Cs, Ns=Ws, beta=&beta;)\n",
    "lqm2.stationary_values()\n",
    "\n",
    "# Simulate the model\n",
    "x, u, w, t = lqm2.compute_sequence(x0, ts_length=300)\n",
    "\n",
    "# Plot of one and two-period debt issuance\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\n",
    "ax1.plot(u[0, :])\n",
    "ax1.set_title('One-period debt issuance')\n",
    "ax1.set_xlabel('Time')\n",
    "ax2.plot(u[1, :])\n",
    "ax2.set_title('Two-period debt issuance')\n",
    "ax2.set_xlabel('Time')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Model with Restructuring\n",
    "\n",
    "This model alters two features of the previous model:\n",
    "\n",
    "1. The maximum horizon of government debt is now extended to a general\n",
    "  *H* periods.  \n",
    "1. The government is able to redesign the maturity structure of debt\n",
    "  every period.  \n",
    "\n",
    "\n",
    "We impose a cost on adjusting issuance of each maturity by amending the\n",
    "payoff function to become:\n",
    "\n",
    "$$\n",
    "T_t^2 + \\sum_{j=0}^{H-1} c_2 (b_{t+j}^{t-1} - b_{t+j+1}^t)^2\n",
    "$$\n",
    "\n",
    "The government&rsquo;s budget constraint is now:\n",
    "\n",
    "$$\n",
    "T_t + \\sum_{j=1}^Hp_{t,t+j} b_{t+j}^t = b_t^{t-1} + \\sum_{j=1}^{H-1} p_{t,t+j} b_{t+j}^{t-1} + G_t\n",
    "$$\n",
    "\n",
    "To map this into the Markov Jump LQ framework, we define state and\n",
    "control variables.\n",
    "\n",
    "Let:\n",
    "\n",
    "$$\n",
    "\\bar b_t = \\begin{bmatrix} b^{t-1}_t \\\\ b^{t-1}_{t+1} \\\\ \\vdots \\\\  b^{t-1}_{t+H-1} \\end{bmatrix} \\hspace{2mm} , \\hspace{2mm} u_t = \\begin{bmatrix} b^{t}_{t+1} \\\\ b^{t}_{t+2} \\\\ \\vdots \\\\  b^{t}_{t+H} \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Thus, $ \\bar b_t $ is the endogenous state (debt issued last period)\n",
    "and $ u_t $ is the control (debt issued today).\n",
    "\n",
    "As before, we will\n",
    "also have the exogenous state $ z_t $, which determines government\n",
    "spending.\n",
    "\n",
    "Therefore, the full state is:\n",
    "\n",
    "$$\n",
    "x_t = \\begin{bmatrix} \\bar b_t \\\\ z_t \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "We also define a vector $ p_t $ that contains the time $ t $\n",
    "price of goods in period $ t + j $:\n",
    "\n",
    "$$\n",
    "p_t = \\begin{bmatrix} p_{t,t+1} \\\\ p_{t,t+2} \\\\ \\vdots \\\\  p_{t,t+H} \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Finally, we define three useful matrices $ S_s, S_x, \\tilde S_x $:\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix} p_{t,t+1} \\\\ p_{t,t+2} \\\\ \\vdots \\\\  p_{t,t+H-1} \\end{bmatrix} = S_s p_t \\text{ where } S_s = \\begin{bmatrix} 1 & 0 & 0 & \\cdots & 0 \\\\ 0 & 1 & 0 & \\cdots & 0 \\\\ \\vdots & & \\ddots & & \\\\ 0 & 0 & \\cdots & 1 & 0  \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix} b^{t-1}_{t+1} \\\\ b^{t-1}_{t+2} \\\\ \\vdots \\\\  b^{t-1}_{t+T-1} \\end{bmatrix} = S_x \\bar b_t  \\text{ where } S_x = \\begin{bmatrix} 0 & 1 & 0 & \\cdots & 0 \\\\ 0 & 0 & 1 & \\cdots & 0 \\\\ \\vdots & &  & \\ddots & \\\\ 0 & 0 & \\cdots & 0 & 1  \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "b^{t-1}_t = \\tilde S_x \\bar b_t \\text{ where } \\tilde S_x = \\begin{bmatrix} 1 & 0 & 0 & \\cdots & 0 \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "In terms of dimensions, the first two matrices defined above are $ (H-1) \\times H $.\n",
    "\n",
    "The last is $ 1 \\times H $\n",
    "\n",
    "We can now write the government&rsquo;s budget constraint in matrix notation.\n",
    "Rearranging the government budget constraint gives:\n",
    "\n",
    "$$\n",
    "T_t = b_t^{t-1} + \\sum_{j=1}^{H-1} p_{t+j}^t b_{t+j}^{t-1} + G_t - \\sum_{j=1}^H p_{t+j}^t b_{t+j}^t\n",
    "$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\n",
    "T_t = \\tilde S_x \\bar b_t + (S_s p_t) \\cdot (S_x \\bar b_t) + U_g z_t - p_t \\cdot u_t\n",
    "$$\n",
    "\n",
    "If we want to write this in terms of the full state, we have:\n",
    "\n",
    "$$\n",
    "T_t = \\begin{bmatrix}  (\\tilde S_x + p_t'S_s'S_x) & Ug \\end{bmatrix} x_t - p_t' u_t\n",
    "$$\n",
    "\n",
    "To simplify the notation, let $ S_t = \\begin{bmatrix}  (\\tilde S_x + p_t&rsquo;S_s&rsquo;S_x) & Ug \\end{bmatrix} $.\n",
    "\n",
    "Then\n",
    "\n",
    "$$\n",
    "T_t = S_t x_t - p_t' u_t\n",
    "$$\n",
    "\n",
    "Therefore\n",
    "\n",
    "$$\n",
    "T_t^2 = x_t' R_t x_t + u_t ' Q_t u_t + 2 u_t'W_t x_t\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "R_t = S_t'S_t , \\hspace{5mm} Q_t = p_t p_t' , \\hspace{5mm} W_t = -p_t S_t\n",
    "$$\n",
    "\n",
    "Because the payoff function also includes the penalty parameter for\n",
    "rescheduling, we have:\n",
    "\n",
    "$$\n",
    "T_t^2 + \\sum_{j=0}^{H-1} c_2 (b_{t+j}^{t-1} - b_{t+j+1}^t)^2 = T_t^2 + c_2(\\bar b_t - u_t)'(\\bar b_t - u_t)\n",
    "$$\n",
    "\n",
    "Because the complete state is $ x_t $ and not $ \\bar b_t $, we\n",
    "rewrite this as:\n",
    "\n",
    "$$\n",
    "T_t^2 + c_2(S_c x_t - u_t)'(S_c x_t - u_t)\n",
    "$$\n",
    "\n",
    "where $ S_c = \\begin{bmatrix} I & 0 \\end{bmatrix} $\n",
    "\n",
    "Multiplying this out gives:\n",
    "\n",
    "$$\n",
    "T_t^2 + c_2 x_t' S_c' S_c x_t - 2c_2 u_t' S_c x_t + c_2 u_t'u_t\n",
    "$$\n",
    "\n",
    "Therefore, with the cost term, we must amend our $ R,Q,W $ matrices\n",
    "as follows:\n",
    "\n",
    "$$\n",
    "R^c_t = R_t + c_2 S_c'S_c\n",
    "$$\n",
    "\n",
    "$$\n",
    "Q^c_t = Q_t + c_2 I\n",
    "$$\n",
    "\n",
    "$$\n",
    "W^c_t = W_t - c_2 S_c\n",
    "$$\n",
    "\n",
    "To finish mapping into the Markov jump LQ setup, we need to construct\n",
    "the law of motion for the full state.\n",
    "\n",
    "This is simpler than in the\n",
    "previous setup, as we now have $ \\bar b_{t+1} = u_t $.\n",
    "\n",
    "Therefore:\n",
    "\n",
    "$$\n",
    "x_{t+1} \\equiv \\begin{bmatrix} \\bar b_{t+1} \\\\ z_{t+1} \\end{bmatrix} = A_t x_t + B u_t + C_t w_{t+1}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "A_t = \\begin{bmatrix} 0 & 0 \\\\ 0 & A_{22,t} \\end{bmatrix} , \\hspace{5mm} B = \\begin{bmatrix} I \\\\ 0 \\end{bmatrix} , \\hspace{5mm} C = \\begin{bmatrix} 0 \\\\ C_{2,t} \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "This completes the mapping into a Markov jump LQ problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model with Restructuring as a Markov Jump Linear Quadratic Control Problem\n",
    "\n",
    "As with the previous model, we can use a function to map the primitives\n",
    "of the model with restructuring into the matrices that the `LQMarkov`\n",
    "class requires:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "def LQ_markov_mapping_restruct(A22, C2, Ug, T, p_t, c=0):\n",
    "\n",
    "    \"\"\"\n",
    "    Function which takes A22, C2, T, p_t, c and returns the\n",
    "    required matrices for the LQMarkov model: A, B, C, R, Q, W\n",
    "    Note, p_t should be a T by 1 matrix\n",
    "    c is the rescheduling cost (a scalar)\n",
    "    This version uses the condensed version of the endogenous state\n",
    "    \"\"\"\n",
    "\n",
    "    # Make sure all matrices can be treated as 2D arrays\n",
    "    A22 = np.atleast_2d(A22)\n",
    "    C2 = np.atleast_2d(C2)\n",
    "    Ug = np.atleast_2d(Ug)\n",
    "    p_t = np.atleast_2d(p_t)\n",
    "\n",
    "    # Find the number of states (z) and shocks (w)\n",
    "    nz, nw = C2.shape\n",
    "\n",
    "    # Create Sx, tSx, Ss, S_t matrices (tSx stands for \\tilde S_x)\n",
    "    Ss = np.hstack((np.eye(T-1), np.zeros((T-1, 1))))\n",
    "    Sx = np.hstack((np.zeros((T-1, 1)), np.eye(T-1)))\n",
    "    tSx = np.zeros((1, T))\n",
    "    tSx[0, 0] = 1\n",
    "\n",
    "    S_t = np.hstack((tSx + p_t.T @ Ss.T @ Sx, Ug))\n",
    "\n",
    "    # Create A, B, C matrices\n",
    "    A_T = np.hstack((np.zeros((T, T)), np.zeros((T, nz))))\n",
    "    A_B = np.hstack((np.zeros((nz, T)), A22))\n",
    "    A = np.vstack((A_T, A_B))\n",
    "\n",
    "    B = np.vstack((np.eye(T), np.zeros((nz, T))))\n",
    "    C = np.vstack((np.zeros((T, nw)), C2))\n",
    "\n",
    "    # Create cost matrix Sc\n",
    "    Sc = np.hstack((np.eye(T), np.zeros((T, nz))))\n",
    "\n",
    "    # Create R_t, Q_t, W_t matrices\n",
    "\n",
    "    R_c = S_t.T @ S_t + c * Sc.T @ Sc\n",
    "    Q_c = p_t @ p_t.T + c * np.eye(T)\n",
    "    W_c = -p_t @ S_t - c * Sc\n",
    "\n",
    "    return A, B, C, R_c, Q_c, W_c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example Model with Restructuring\n",
    "\n",
    "As an example of the model with restructuring, consider this model\n",
    "where $ H = 3 $.\n",
    "\n",
    "We will assume that there are two Markov states, one with a\n",
    "flatter yield curve, and one with a steeper yield curve.\n",
    "\n",
    "In state 1,\n",
    "prices are:\n",
    "\n",
    "$$\n",
    "p^1_{t,t+1} = 0.9695 \\hspace{2mm} , \\hspace{2mm} p^1_{t,t+2} = 0.902 \\hspace{2mm} , \\hspace{2mm} p^1_{t,t+3} = 0.8369\n",
    "$$\n",
    "\n",
    "and in state 2, prices are:\n",
    "\n",
    "$$\n",
    "p^2_{t,t+1} = 0.9295 \\hspace{2mm} , \\hspace{2mm} p^2_{t,t+2} = 0.902 \\hspace{2mm} , \\hspace{2mm} p^2_{t,t+3} = 0.8769\n",
    "$$\n",
    "\n",
    "We will assume the same transition matrix and $ G_t $ process as\n",
    "above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# New model parameters\n",
    "H = 3\n",
    "p1 = np.array([[0.9695], [0.902], [0.8369]])\n",
    "p2 = np.array([[0.9295], [0.902], [0.8769]])\n",
    "Pi = np.array([[0.9, 0.1], [0.1, 0.9]])\n",
    "\n",
    "# Put penalty on different issuance across maturities\n",
    "c2 = 0.5\n",
    "\n",
    "A1, B1, C1, R1, Q1, W1 = LQ_markov_mapping_restruct(A22, C_2, Ug, H, p1, c2)\n",
    "A2, B2, C2, R2, Q2, W2 = LQ_markov_mapping_restruct(A22, C_2, Ug, H, p2, c2)\n",
    "\n",
    "# Small penalties on debt required to implement no-Ponzi scheme\n",
    "R1[0, 0] = R1[0, 0] + 1e-9\n",
    "R1[1, 1] = R1[1, 1] + 1e-9\n",
    "R1[2, 2] = R1[2, 2] + 1e-9\n",
    "R2[0, 0] = R2[0, 0] + 1e-9\n",
    "R2[1, 1] = R2[1, 1] + 1e-9\n",
    "R2[2, 2] = R2[2, 2] + 1e-9\n",
    "\n",
    "# Construct lists of matrices\n",
    "As = [A1, A2]\n",
    "Bs = [B1, B2]\n",
    "Cs = [C1, C2]\n",
    "Rs = [R1, R2]\n",
    "Qs = [Q1, Q2]\n",
    "Ws = [W1, W2]\n",
    "\n",
    "# Construct and solve the model using the LQMarkov class\n",
    "lqm3 = qe.LQMarkov(&Pi;, Qs, Rs, As, Bs, Cs=Cs, Ns=Ws, beta=&beta;)\n",
    "lqm3.stationary_values()\n",
    "\n",
    "x0 = np.array([[5000, 5000, 5000, 1, 10]])\n",
    "x, u, w, t = lqm3.compute_sequence(x0, ts_length=300)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Plots of different maturities debt issuance\n",
    "\n",
    "fig, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize=(16, 4))\n",
    "ax1.plot(u[0, :])\n",
    "ax1.set_title('One-period debt issuance')\n",
    "ax1.set_xlabel('Time')\n",
    "ax2.plot(u[1, :])\n",
    "ax2.set_title('Two-period debt issuance')\n",
    "ax2.set_xlabel('Time')\n",
    "ax3.plot(u[2, :])\n",
    "ax3.set_title('Three-period debt issuance')\n",
    "ax3.set_xlabel('Time')\n",
    "ax4.plot(u[0, :] + u[1, :] + u[2, :])\n",
    "ax4.set_title('Total debt issuance')\n",
    "ax4.set_xlabel('Time')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Plot share of debt issuance that is short-term\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "ax.plot((u[0, :] / (u[0, :] + u[1, :] + u[2, :])))\n",
    "ax.set_title('One-period debt issuance share')\n",
    "ax.set_xlabel('Time')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "filename": "tax_smoothing_2.rst",
  "kernelspec": {
   "display_name": "Python",
   "language": "python3",
   "name": "python3"
  },
  "title": "How to Pay for a War: Part 2"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}