1.5Scipy：高级科学计算.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "> 作者: Adrien Chauve, Andre Espaze, Emmanuelle Gouillart, Gaël Varoquaux, Ralf Gommers\n",
    "\n",
    "> 原文: http://www.scipy-lectures.org/intro/scipy.html\n",
    "\n",
    "---\n",
    "\n",
    "**Scipy**\n",
    "\n",
    "`scipy`包含许多专注于科学计算中的常见问题的工具箱。它的子模块对应于不同的应用，比如插值、积分、优化、图像处理、统计和特殊功能等。\n",
    "\n",
    "`scipy`可以与其他标准科学计算包相对比，比如GSL (C和C++的GNU科学计算包), 或者Matlab的工具箱。`scipy`是Python中科学程序的核心程序包；这意味着有效的操作`numpy`数组，因此，numpy和scipy可以一起工作。\n",
    "\n",
    "在实现一个程序前，有必要确认一下需要的数据处理方式是否已经在scipy中实现了。作为非专业程序员，科学家通常倾向于**重新发明轮子**，这产生了小玩具、不优化、很难分享以及不可以维护的代码。相反，scipy的程序是优化并且测试过的，因此应该尽可能使用。\n",
    "\n",
    "\n",
    "**警告** 这个教程根本不是数值计算的介绍。因为列举scipy的不同子模块和功能将会是非常枯燥的，相反我们将聚焦于列出一些例子，给出如何用scipy进行科学计算的大概思路。\n",
    "\n",
    "scipy是由针对特定任务的子模块组成的：\n",
    "\n",
    "<table border=\"1\" class=\"docutils\">\n",
    "<colgroup>\n",
    "<col width=\"36%\">\n",
    "<col width=\"64%\">\n",
    "</colgroup>\n",
    "<tbody valign=\"top\">\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/cluster.html#module-scipy.cluster\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.cluster</span></tt></a></td>\n",
    "<td>向量计算 / Kmeans</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/constants.html#module-scipy.constants\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.constants</span></tt></a></td>\n",
    "<td>物理和数学常量</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/fftpack.html#module-scipy.fftpack\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.fftpack</span></tt></a></td>\n",
    "<td>傅里叶变换</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/integrate.html#module-scipy.integrate\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.integrate</span></tt></a></td>\n",
    "<td>积分程序</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/interpolate.html#module-scipy.interpolate\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.interpolate</span></tt></a></td>\n",
    "<td>插值</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/io.html#module-scipy.io\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.io</span></tt></a></td>\n",
    "<td>数据输入和输出</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/linalg.html#module-scipy.linalg\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.linalg</span></tt></a></td>\n",
    "<td>线性代数程序</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/ndimage.html#module-scipy.ndimage\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.ndimage</span></tt></a></td>\n",
    "<td>n-维图像包</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/odr.html#module-scipy.odr\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.odr</span></tt></a></td>\n",
    "<td>正交距离回归</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/optimize.html#module-scipy.optimize\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.optimize</span></tt></a></td>\n",
    "<td>优化</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/signal.html#module-scipy.signal\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.signal</span></tt></a></td>\n",
    "<td>信号处理</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/sparse.html#module-scipy.sparse\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.sparse</span></tt></a></td>\n",
    "<td>稀疏矩阵</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/spatial.html#module-scipy.spatial\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.spatial</span></tt></a></td>\n",
    "<td>空间数据结构和算法</td>\n",
    "</tr>\n",
    "<tr class=\"row-even\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/special.html#module-scipy.special\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.special</span></tt></a></td>\n",
    "<td>一些特殊数学函数</td>\n",
    "</tr>\n",
    "<tr class=\"row-odd\"><td><a class=\"reference external\" href=\"http://docs.scipy.org/doc/scipy/reference/stats.html#module-scipy.stats\" title=\"(in SciPy v0.19.0)\"><tt class=\"xref py py-mod docutils literal\"><span class=\"pre\">scipy.stats</span></tt></a></td>\n",
    "<td>统计</td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>\n",
    "\n",
    "\n",
    "\n",
    "他们全都依赖于[numpy](http://docs.scipy.org/doc/numpy/reference/index.html#numpy), 但是大多数是彼此独立的。导入Numpy和Scipy的标准方式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from scipy import stats  # 其他的子模块类似"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`scipy`的主要命名空间通常包含的函数其实是numpy（试一下`scipy.cos`其实是`np.cos`) 。这些函数的暴露只是因为历史原因；通常没有必要在你的代码中使用`import scipy`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.1 文件输入/输出：[scipy.io](http://docs.scipy.org/doc/scipy/reference/io.html#scipy.io)\n",
    "\n",
    "载入和保存matlab文件："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import io as spio\n",
    "\n",
    "a = np.ones((3, 3))\n",
    "spio.savemat('file.mat', {'a': a}) # savemat expects a dictionary\n",
    "data = spio.loadmat('file.mat', struct_as_record=True)\n",
    "data['a']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "from scipy import misc\n",
    "misc.imread('fname.png')\n",
    "# Matplotlib也有类似的方法\n",
    "import matplotlib.pyplot as plt\n",
    "plt.imread('fname.png')\n",
    "```\n",
    "更多请见：\n",
    "\n",
    "* 加载文本文件：[numpy.loadtxt()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html#numpy.loadtxt)/[numpy.savetxt()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html#numpy.savetxt)\n",
    "* 智能加载文本/csv文件：[numpy.genfromtxt()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html#numpy.genfromtxt)/numpy.recfromcsv()\n",
    "* 快速有效，但是针对numpy的二进制格式：[numpy.save()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html#numpy.save)/[numpy.load()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.load.html#numpy.load)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.2 特殊函数：[scipy.special](http://docs.scipy.org/doc/scipy/reference/special.html#scipy.special)\n",
    "\n",
    "特殊函数是超验函数。[scipy.special](http://docs.scipy.org/doc/scipy/reference/special.html#scipy.special)模块的文档字符串写的很详细，因此我们不会在这里列出所有的函数。常用的一些函数如下：\n",
    "\n",
    "* 贝塞尔函数，比如`scipy.special.jn()` (第n个整型顺序的贝塞尔函数)\n",
    "* 椭圆函数 (`scipy.special.ellipj()` Jacobian椭圆函数, ...)\n",
    "* Gamma 函数: scipy.special.gamma(), 也要注意 `scipy.special.gammaln()` 将给出更高准确数值的 Gamma的log。\n",
    "* Erf, 高斯曲线的面积：scipy.special.erf()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.3 线性代数操作：[scipy.linalg](http://docs.scipy.org/doc/scipy/reference/linalg.html#scipy.linalg)\n",
    "\n",
    "[scipy.linalg](http://docs.scipy.org/doc/scipy/reference/linalg.html#scipy.linalg) 模块提供了标准的线性代数操作，这依赖于底层的高效实现（BLAS、LAPACK）。\n",
    "* [scipy.linalg.det()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.det.html#scipy.linalg.det) 函数计算方阵的行列式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import linalg\n",
    "arr = np.array([[1, 2],\n",
    "                [3, 4]])\n",
    "linalg.det(arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "arr = np.array([[3, 2],\n",
    "                 [6, 4]])\n",
    "linalg.det(arr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "linalg.det(np.ones((3, 4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [scipy.linalg.inv()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.inv.html#scipy.linalg.inv) 函数计算逆方阵："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "arr = np.array([[1, 2],\n",
    "                 [3, 4]])\n",
    "iarr = linalg.inv(arr)\n",
    "iarr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "np.allclose(np.dot(arr, iarr), np.eye(2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后计算逆奇异矩阵（行列式为0）将抛出`LinAlgError` :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "arr = np.array([[3, 2],\n",
    "                 [6, 4]])\n",
    "linalg.inv(arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 还有更多高级的操作，奇异值分解（SVD）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "arr = np.arange(9).reshape((3, 3)) + np.diag([1, 0, 1])\n",
    "uarr, spec, vharr = linalg.svd(arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果的数组频谱是："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "spec    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原始矩阵可以用`svd`和`np.dot`矩阵相乘的结果重新获得："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sarr = np.diag(spec)\n",
    "svd_mat = uarr.dot(sarr).dot(vharr)\n",
    "np.allclose(svd_mat, arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SVD常被用于统计和信号处理。其他标准分解 (QR, LU, Cholesky, Schur), 以及线性系统的求解器，也可以在[scipy.linalg](http://docs.scipy.org/doc/scipy/reference/linalg.html#scipy.linalg)中找到。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.4 快速傅立叶变换：[scipy.fftpack](http://docs.scipy.org/doc/scipy/reference/fftpack.html#scipy.fftpack)\n",
    "\n",
    "[scipy.fftpack](http://docs.scipy.org/doc/scipy/reference/fftpack.html#scipy.fftpack) 模块允许计算快速傅立叶变换。例子，一个（有噪音）的信号输入是这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "time_step = 0.02\n",
    "period = 5.\n",
    "time_vec = np.arange(0, 20, time_step)\n",
    "sig = np.sin(2 * np.pi / period * time_vec) + \\\n",
    "       0.5 * np.random.randn(time_vec.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "观察者并不知道信号的频率，只知道抽样时间步骤的信号`sig`。假设信号来自真实的函数，因此傅立叶变换将是对称的。[scipy.fftpack.fftfreq()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.fftpack.fftfreq.html#scipy.fftpack.fftfreq) 函数将生成样本序列，而[scipy.fftpack.fft()]()将计算快速傅立叶变换："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import fftpack\n",
    "sample_freq = fftpack.fftfreq(sig.size, d=time_step)\n",
    "sig_fft = fftpack.fft(sig)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为生成的幂是对称的，寻找频率只需要使用频谱为正的部分："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "pidxs = np.where(sample_freq > 0)\n",
    "freqs = sample_freq[pidxs]\n",
    "power = np.abs(sig_fft)[pidxs]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![png](http://scipy-lectures.github.io/_images/fftpack_frequency.png)](http://scipy-lectures.github.io/plot_directive/pyplots/fftpack_frequency.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "寻找信号频率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "freq = freqs[power.argmax()]\n",
    "np.allclose(freq, 1./period)  # 检查是否找到了正确的频率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在高频噪音将从傅立叶转换过的信号移除："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sig_fft[np.abs(sample_freq) > freq] = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成的过滤过的信号可以用[scipy.fftpack.ifft()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.fftpack.ifft.html#scipy.fftpack.ifft)函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "main_sig = fftpack.ifft(sig_fft)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.figure()\n",
    "plt.plot(time_vec, sig)\n",
    "plt.plot(time_vec, main_sig, linewidth=3)\n",
    "plt.xlabel('Time [s]')\n",
    "plt.ylabel('Amplitude')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "[numpy.fft](http://docs.scipy.org/doc/numpy/reference/routines.fft.html#numpy.fft)\n",
    "\n",
    "Numpy也有一个FFT(numpy.fft)实现。但是，通常scipy的实现更受欢迎，因为，他使用更高效的底层实现。 \n",
    "\n",
    "---\n",
    "\n",
    "---\n",
    "\n",
    "**实例：寻找粗略周期**\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/periodicity_finder_00.png)](http://scipy-lectures.github.io/plot_directive/intro/solutions/periodicity_finder.py)\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/periodicity_finder_01.png)](http://scipy-lectures.github.io/plot_directive/intro/solutions/periodicity_finder.py)\n",
    "\n",
    "---\n",
    "\n",
    "---\n",
    "\n",
    "**实例：高斯图片模糊**\n",
    "\n",
    "弯曲:\n",
    "\n",
    "$f_1(t) = \\int dt'\\, K(t-t') f_0(t')$\n",
    "\n",
    "$\\tilde{f}_1(\\omega) = \\tilde{K}(\\omega) \\tilde{f}_0(\\omega)$\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/image_blur.png)](http://scipy-lectures.github.io/plot_directive/intro/solutions/image_blur.py)\n",
    "\n",
    "---\n",
    "\n",
    "**练习：月球登陆图片降噪**\n",
    "\n",
    "![](http://scipy-lectures.github.io/_images/moonlanding.png)\n",
    "\n",
    "1. 检查提供的图片moonlanding.png，图片被周期噪音污染了。在这个练习中，我们的目的是用快速傅立叶变换清除噪音。\n",
    "\n",
    "2. 用`pylab.imread()`加载图片。\n",
    "\n",
    "3. 寻找并使用在[scipy.fftpack](http://docs.scipy.org/doc/scipy/reference/fftpack.html#scipy.fftpack)中的2-D FFT函数，绘制图像的频谱（傅立叶变换）。在可视化频谱时是否遇到了麻烦？如果有的话，为什么？\n",
    "\n",
    "4. 频谱由高频和低频成分构成。噪音被包含在频谱的高频部分，因此将那些部分设置为0（使用数组切片）。\n",
    "\n",
    "5. 应用逆傅立叶变换来看一下结果图片。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.5 优化及拟合：[scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html#scipy.optimize)\n",
    "\n",
    "优化是寻找最小化或等式的数值解的问题。\n",
    "\n",
    "[scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html#scipy.optimize) 模块提供了函数最小化（标量或多维度）、曲线拟合和求根的有用算法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import optimize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**寻找标量函数的最小值**\n",
    "\n",
    "让我们定义下面的函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return x**2 + 10*np.sin(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "绘制它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.arange(-10, 10, 0.1)\n",
    "plt.plot(x, f(x)) \n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数在-1.3附近有一个全局最小并且在3.8有一个局部最小。\n",
    "\n",
    "找到这个函数的最小值的常用有效方式是从给定的初始点开始进行一个梯度下降。BFGS算法是这样做的较好方式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "optimize.fmin_bfgs(f, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个方法的一个可能问题是，如果这个函数有一些局部最低点，算法可能找到这些局部最低点而不是全局最低点，这取决于初始点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "optimize.fmin_bfgs(f, 3, disp=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们不知道全局最低点，并且使用其临近点来作为初始点，那么我们需要付出昂贵的代价来获得全局最优。要找到全局最优点，最简单的算法是暴力算法，算法中会评估给定网格内的每一个点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "grid = (-10, 10, 0.1)\n",
    "xmin_global = optimize.brute(f, (grid,))\n",
    "xmin_global"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于更大的网格，[scipy.optimize.brute()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brute.html#scipy.optimize.brute) 变得非常慢。[scipy.optimize.anneal()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.anneal.html#scipy.optimize.anneal) 提供了一个替代的算法，使用模拟退火。对于不同类型的全局优化问题存在更多的高效算法，但是这超出了`scipy`的范畴。[OpenOpt](http://openopt.org/Welcome)、[IPOPT](https://github.com/xuy/pyipopt)、[PyGMO](http://pagmo.sourceforge.net/pygmo/index.html)和[PyEvolve](http://pyevolve.sourceforge.net/)是关于全局优化的一些有用的包。\n",
    "\n",
    "要找出局部最低点，让我们用[scipy.optimize.fminbound](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fminbound.html#scipy.optimize.fminbound)将变量限制在(0,10)区间："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "xmin_local = optimize.fminbound(f, 0, 10)    \n",
    "xmin_local"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**注**：寻找函数的最优解将在高级章节中：[数学优化：寻找函数的最优解](http://scipy-lectures.github.io/advanced/mathematical_optimization/index.html#mathematical-optimization)详细讨论。\n",
    "\n",
    "---\n",
    "\n",
    "**寻找标量函数的根**\n",
    "\n",
    "要寻找上面函数f的根，比如`f(x)=0`的一个点，我们可以用比如[scipy.optimize.fsolve()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html#scipy.optimize.fsolve)："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "root = optimize.fsolve(f, 1)  # 我们的最初猜想是1\n",
    "root"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意只找到一个根。检查`f`的图发现在-2.5左右还有应该有第二个根。通过调整我们最初的猜想，我们可以发现正确的值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "root2 = optimize.fsolve(f, -2.5)\n",
    "root2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**曲线拟合**\n",
    "\n",
    "假设我们有来自`f`的样例数据，带有一些噪音："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "xdata = np.linspace(-10, 10, num=20)\n",
    "ydata = f(xdata) + np.random.randn(xdata.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，如果我们知道这些sample数据来自的函数（这个案例中是$x^2 + sin(x)$）的函数形式，而不知道每个数据项的系数，那么我们可以用最小二乘曲线拟合在找到这些系数。首先，我们需要定义函数来拟合："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def f2(x, a, b):\n",
    "    return a*x**2 + b*np.sin(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后我们可以使用[scipy.optimize.curve_fit()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html#scipy.optimize.curve_fit)来找到`a`和`b`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "guess = [2, 2]\n",
    "params, params_covariance = optimize.curve_fit(f2, xdata, ydata, guess)\n",
    "params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们找到了`f`的最优解和根，并且用曲线去拟合它，我们将这些结果整合在一个图中："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![](http://scipy-lectures.github.io/_images/scipy_optimize_example2.png)](http://scipy-lectures.github.io/plot_directive/pyplots/scipy_optimize_example2.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**注**：在Scipy >= 0.11中，包含所有最小值和寻找根的算法的统一接口：[scipy.optimize.minimize()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize)、 [scipy.optimize.minimize_scalar()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize_scalar.html#scipy.optimize.minimize_scalar)和 [scipy.optimize.root()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.root.html#scipy.optimize.root)。他们允许通过`method`关键词容易的比较多种算法。\n",
    "\n",
    "---\n",
    "\n",
    "你可以在[scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html#scipy.optimize)中找到对于多维度问题有相同功能的算法。\n",
    "\n",
    "---\n",
    "\n",
    "**练习：温度数据的曲线拟合**\n",
    "\n",
    "下面是从1月开始阿拉斯加每个月的温度极值（摄氏度）：\n",
    "\n",
    "最大值:  17,  19,  21,  28,  33,  38, 37,  37,  31,  23,  19,  18\n",
    "\n",
    "最小值: -62, -59, -56, -46, -32, -18, -9, -13, -25, -46, -52, -58\n",
    "\n",
    "1. 绘制这些温度极值。\n",
    "2. 定义一个函数，可以描述温度的最大值和最小值。提示：这个函数的周期是一年。提示：包含时间偏移。\n",
    "3. 用[scipy.optimize.curve_fit()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html#scipy.optimize.curve_fit)拟合这个函数与数据。\n",
    "4. 绘制结果。这个拟合合理吗？如果不合理，为什么？\n",
    "5. 最低温度和最高温度的时间偏移是否与拟合一样精确？\n",
    "\n",
    "---\n",
    "\n",
    "---\n",
    "**练习：2-D 最小值**\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/scipy_optimize_sixhump.png)](http://scipy-lectures.github.io/plot_directive/pyplots/scipy_optimize_sixhump.py)\n",
    "\n",
    "六峰驼背函数：\n",
    "![](http://scipy-lectures.github.io/_images/math/a15fd87c4934c2530d57ccae635f1e53791a5187.png)\n",
    "\n",
    "有多个全局和局部最低点。找到这个函数的全局最低点。\n",
    "\n",
    "提示：\n",
    "\n",
    "* 变量可以被限定在-2 < x < 2 和 -1 < y < 1。\n",
    "* 用[numpy.meshgrid()](http://docs.scipy.org/doc/numpy/reference/generated/numpy.meshgrid.html#numpy.meshgrid) 和 pylab.imshow() 来从视觉上来寻找区域。\n",
    "* [scipy.optimize.fmin_bfgs()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_bfgs.html#scipy.optimize.fmin_bfgs) 或者另一个多维最小化。\n",
    "多几个全局最小值，那些点上的函数值十多少？如果最初的猜测是$(x, y) = (0, 0)$会怎样?\n",
    "\n",
    "---\n",
    "\n",
    "看一下[非线性最小二乘曲线拟合：地形机载激光雷达数据中的点抽取](http://scipy-lectures.github.io/intro/summary-exercises/optimize-fit.html#summary-exercise-optimize)练习的总结，以及更高及的例子。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.6. 统计和随机数：[scipy.stats](http://docs.scipy.org/doc/scipy/reference/stats.html#scipy.stats)\n",
    "\n",
    "[scipy.stats](http://docs.scipy.org/doc/scipy/reference/stats.html#scipy.stats)模块包含统计工具和随机过程的概率描述。在`numpy.random`中可以找到多个随机数生成器。\n",
    "\n",
    "### 1.5.6.1 直方图和概率密度函数\n",
    "\n",
    "给定随机过程的观察值，它们的直方图是随机过程的PDF（概率密度函数）的估计值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.random.normal(size=1000)\n",
    "bins = np.arange(-4, 5)\n",
    "bins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "histogram = np.histogram(a, bins=bins, normed=True)[0]\n",
    "bins = 0.5*(bins[1:] + bins[:-1])\n",
    "bins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import stats\n",
    "\n",
    "\n",
    "b = stats.norm.pdf(bins)  # norm 是一种分布\n",
    "plt.plot(bins, histogram)\n",
    "plt.plot(bins, b)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们知道随机过程属于特定的随机过程家族，比如正态过程，我们可以做一个观察值的最大可能性拟合，来估计潜在分布的参数。这里我们用随机过程拟合观察数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "loc, std = stats.norm.fit(a)\n",
    "loc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "**练习：概率分布**\n",
    "\n",
    "用shape参数为1的gamma分布生成1000个随机数，然后绘制那些样本的直方图。你可以在顶部绘制pdf（应该会匹配）吗？\n",
    "\n",
    "额外信息：这些分布都有一些有用的方法。读一下文档字符串或者用IPython tab 完成来研究这些方法。你可以用在你的随机变量上使用`fit`方法来找回shape参数1吗？\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.6.2 百分位数\n",
    "\n",
    "中数是有一半值在其上一半值在其下的值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "np.median(a) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "中数也被称为百分位数50，因为50%的观察值在它之下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "stats.scoreatpercentile(a, 50) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样，我们也能计算百分位数90："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "stats.scoreatpercentile(a, 90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "百分位数是CDF的估计值：累积分布函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.6.3 统计检验\n",
    "\n",
    "统计检验是一个决策指示器。例如，如果我们有两组观察值，我们假设他们来自于高斯过程，我们可以用T检验来决定这两组观察值是不是显著不同："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.random.normal(0, 1, size=100)\n",
    "b = np.random.normal(1, 1, size=10)\n",
    "stats.ttest_ind(a, b)   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "生成的结果由以下内容组成：\n",
    "\n",
    "* T 统计值：一个值，符号与两个随机过程的差异成比例，大小与差异的程度有关。\n",
    "* p 值：两个过程相同的概率。如果它接近1，那么这两个过程几乎肯定是相同的。越接近于0，越可能这两个过程有不同的平均数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.7 插值：[scipy.interpolate](http://docs.scipy.org/doc/scipy/reference/interpolate.html#scipy.interpolate)\n",
    "\n",
    "[scipy.interpolate](http://docs.scipy.org/doc/scipy/reference/interpolate.html#scipy.interpolate)对从实验数据中拟合函数是非常有用的，因此，评估没有测量过的点。这个模块是基于[netlib](http://www.netlib.org/)项目的[Fortran子程序 FITPACK](http://www.netlib.org/dierckx/index.html)\n",
    "\n",
    "假想一个接近sine函数的实验数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "measured_time = np.linspace(0, 1, 10)\n",
    "noise = (np.random.random(10)*2 - 1) * 1e-1\n",
    "measures = np.sin(2 * np.pi * measured_time) + noise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[scipy.interpolate.interp1d](http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html#scipy.interpolate.interp1d)类可以建立一个线性插值函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy.interpolate import interp1d\n",
    "linear_interp = interp1d(measured_time, measures)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`scipy.interpolate.linear_interp`实例需要评估感兴趣的时间点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "computed_time = np.linspace(0, 1, 50)\n",
    "linear_results = linear_interp(computed_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过提供可选的参数`kind`也可以选择进行立方插值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "cubic_interp = interp1d(measured_time, measures, kind='cubic')\n",
    "cubic_results = cubic_interp(computed_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在结果可以被整合为下面的Matplotlib图片：\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/scipy_interpolation.png)](http://scipy-lectures.github.io/plot_directive/pyplots/scipy_interpolation.py)\n",
    "\n",
    "[scipy.interpolate.interp2d](http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp2d.html#scipy.interpolate.interp2d) 与[scipy.interpolate.interp1d](http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html#scipy.interpolate.interp1d)类似，但是是用于2-D数组。注意对于`interp`家族，计算的时间点必须在测量时间段之内。看一下[Sprogø气象站的最大风速预测的总结练习](暂缺)，了解更详细的spline插值实例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.8 数值积分：[scipy.integrate]()\n",
    "\n",
    "[scipy.integrate.quad()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.quad.html#scipy.integrate.quad)是最常见的积分程序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy.integrate import quad\n",
    "res, err = quad(np.sin, 0, np.pi/2)\n",
    "np.allclose(res, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "np.allclose(err, 1 - res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其他的积分程序可以在`fixed_quad`、 `quadrature`、`romberg`中找到。\n",
    "\n",
    "[scipy.integrate](http://docs.scipy.org/doc/scipy/reference/integrate.html#scipy.integrate) 可提供了常微分公式(ODE)的特色程序。特别的，[scipy.integrate.odeint()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html#scipy.integrate.odeint) 是使用LSODA（Livermore Solver for Ordinary Differential equations with Automatic method switching for stiff and non-stiff problems）的通用积分器，更多细节请见[ODEPACK Fortran 库](http://people.sc.fsu.edu/~jburkardt/f77_src/odepack/odepack.html)。\n",
    "\n",
    "`odeint`解决如下形式的第一顺序ODE系统：\n",
    "\n",
    "$dy/dt = rhs(y1, y2, .., t0,...)$\n",
    "\n",
    "作为一个介绍，让我们解一下在初始条件下$y(t=0) = 1$，这个常微分公式$dy/dt = -2y$在$t = 0..4$时的值。首先，这个函数计算定义位置需要的导数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def calc_derivative(ypos, time, counter_arr):\n",
    "    counter_arr += 1\n",
    "    return -2 * ypos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "添加了一个额外的参数`counter_arr`用来说明这个函数可以在一个时间步骤被调用多次，直到收敛。计数器数组定义如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "counter = np.zeros((1,), dtype=np.uint16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在计算轨迹线："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy.integrate import odeint\n",
    "time_vec = np.linspace(0, 4, 40)\n",
    "yvec, info = odeint(calc_derivative, 1, time_vec,\n",
    "                    args=(counter,), full_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此，导数函数被调用了40多次（即时间步骤数）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "前十个时间步骤的累积循环数，可以用如下方式获得："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "info['nfe'][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意，求解器对于首个时间步骤需要更多的循环。导数答案`yvec`可以画出来：\n",
    "\n",
    "[![](http://scipy-lectures.github.io/_images/odeint_introduction.png)](http://scipy-lectures.github.io/plot_directive/pyplots/odeint_introduction.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "阻尼弹簧重物振子（二阶振荡器）是使用[scipy.integrate.odeint()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html#scipy.integrate.odeint)的另一个例子。链接到弹簧的重物的位置服从二阶常微分方程$y'' + 2 eps wo  y' + wo^2 y = 0$，其中$wo^2 = k/m$ 弹簧的常数为k, m是重物质量，$eps=c/(2 m wo)$，c是阻尼系数。例如，我们选择如下参数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "mass = 0.5  # kg\n",
    "kspring = 4  # N/m\n",
    "cviscous = 0.4  # N s/m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此系统将是欠阻尼的，因为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "eps = cviscous / (2 * mass * np.sqrt(kspring/mass))\n",
    "eps < 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于[scipy.integrate.odeint()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.odeint.html#scipy.integrate.odeint)求解器，二阶等式需要被变换为系统内向量$Y=(y, y')$的两个一阶等式。为了方便，定义$nu = 2 eps * wo = c / m$和$om = wo^2 = k/m$："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "nu_coef = cviscous / mass\n",
    "om_coef = kspring / mass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此函数将计算速度和加速度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def calc_deri(yvec, time, nuc, omc):\n",
    "    return (yvec[1], -nuc * yvec[1] - omc * yvec[0])\n",
    "\n",
    "time_vec = np.linspace(0, 10, 100)\n",
    "yarr = odeint(calc_deri, (1, 0), time_vec, args=(nu_coef, om_coef))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下的Matplotlib图片显示了最终的位置和速度：\n",
    "[![](http://scipy-lectures.github.io/_images/odeint_damped_spring_mass.png)](http://scipy-lectures.github.io/plot_directive/pyplots/odeint_damped_spring_mass.py)\n",
    "\n",
    "在Sicpy中没有偏微分方程（PDE）求解器。存在其他求解PDE的Python包，比如[fipy](http://www.ctcms.nist.gov/fipy/)或[SfePy](http://code.google.com/p/sfepy/)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.9 信号处理：[scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html#scipy.signal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [scipy.signal.detrend()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.detrend.html#scipy.signal.detrend): 从信号中删除线性趋势："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import signal\n",
    "\n",
    "t = np.linspace(0, 5, 100)\n",
    "x = t + np.random.normal(size=100)\n",
    "\n",
    "plt.plot(t, x, linewidth=3)\n",
    "plt.plot(t, signal.detrend(x), linewidth=3)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [scipy.signal.resample()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.resample.html#scipy.signal.resample): 用FFT从信号中抽出n个点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "t = np.linspace(0, 5, 100)\n",
    "x = np.sin(t)\n",
    "\n",
    "plt.plot(t, x, linewidth=3)\n",
    "plt.plot(t[::2], signal.resample(x, 50), 'ko')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html#scipy.signal) 有许多窗口函数：[scipy.signal.hamming()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.hamming.html#scipy.signal.hamming), [scipy.signal.bartlett()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.bartlett.html#scipy.signal.bartlett), [scipy.signal.blackman()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.blackman.html#scipy.signal.blackman)...\n",
    "- [scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html#scipy.signal) 有滤镜 (中位数滤镜[scipy.signal.medfilt()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.medfilt.html#scipy.signal.medfilt), Wiener[scipy.signal.wiener()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.wiener.html#scipy.signal.wiener)), 但是我们将在图片部分讨论这些。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.10 图像处理：scipy.ndimage\n",
    "\n",
    "scipy中专注于专注于图像处理的模块是scipy.ndimage。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import ndimage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "图像处理程序可以根据他们进行的处理来分类。\n",
    "\n",
    "### 1.5.10.1 图像的几何变换\n",
    "\n",
    "改变原点，解析度，.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import misc\n",
    "\n",
    "face = misc.face(gray=True)\n",
    "shifted_face = ndimage.shift(face, (50, 50))\n",
    "shifted_face2 = ndimage.shift(face, (50, 50), mode='nearest')\n",
    "rotated_face = ndimage.rotate(face, 30)\n",
    "cropped_face = face[50:-50, 50:-50]\n",
    "zoomed_face = ndimage.zoom(face, 2)\n",
    "zoomed_face.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://www.scipy-lectures.org/_images/face_transforms.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "plt.subplot(151)\n",
    "plt.imshow(shifted_face, cmap=plt.cm.gray)\n",
    "plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.10.2 图像滤波器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy import misc\n",
    "\n",
    "\n",
    "face = misc.face(gray=True)\n",
    "face = face[:512, -512:]  # crop out square on right\n",
    "import numpy as np\n",
    "noisy_face = np.copy(face).astype(np.float)\n",
    "noisy_face += face.std() * 0.5 * np.random.standard_normal(face.shape)\n",
    "blurred_face = ndimage.gaussian_filter(noisy_face, sigma=3)\n",
    "median_face = ndimage.median_filter(noisy_face, size=5)\n",
    "from scipy import signal\n",
    "wiener_face = signal.wiener(noisy_face, (5, 5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://www.scipy-lectures.org/_images/filtered_face.png)\n",
    "在[scipy.ndimage.filters](http://docs.scipy.org/doc/scipy/reference/ndimage.html#scipy.ndimage.filters) 和 [scipy.signal](http://docs.scipy.org/doc/scipy/reference/signal.html#scipy.signal) 有更多应用于图像的滤波器。\n",
    "\n",
    "---\n",
    "练习\n",
    "\n",
    "比较不同过滤后图像的条形图\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.10.3 数学形态学\n",
    "\n",
    "数学形态学是集合理论分支出来的一个数学理论。它刻画并转换几何结构。特别是二元的图像（黑白）可以用这种理论来转换：被转换的集合是临近非零值像素的集合。这个理论也可以被扩展到灰度值图像。\n",
    "![](http://scipy-lectures.github.io/_images/morpho_mat1.png)\n",
    "\n",
    "初级数学形态学操作使用结构化的元素，以便修改其他几何结构。\n",
    "\n",
    "首先让我们生成一个结构化元素。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "el = ndimage.generate_binary_structure(2, 1)\n",
    "el"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "el.astype(np.int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 腐蚀"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.zeros((7,7), dtype=np.int)\n",
    "a[1:6, 2:5] = 1\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "ndimage.binary_erosion(a).astype(a.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "#腐蚀移除了比结构小的对象\n",
    "ndimage.binary_erosion(a, structure=np.ones((5,5))).astype(a.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 扩张 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.zeros((5, 5))\n",
    "a[2, 2] = 1\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "ndimage.binary_dilation(a).astype(a.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 开启"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.zeros((5,5), dtype=np.int)\n",
    "a[1:4, 1:4] = 1; a[4, 4] = 1\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# 开启移除了小对象\n",
    "ndimage.binary_opening(a, structure=np.ones((3,3))).astype(np.int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "# 开启也可以平滑拐角\n",
    "ndimage.binary_opening(a).astype(np.int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 闭合: `ndimage.binary_closing`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "练习\n",
    "\n",
    "验证一下开启相当于先腐蚀再扩张。\n",
    "\n",
    "---\n",
    "\n",
    "开启操作移除小的结构，而关闭操作填满了小洞。因此这些用来”清洗“图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.zeros((50, 50))\n",
    "a[10:-10, 10:-10] = 1\n",
    "a += 0.25*np.random.standard_normal(a.shape)\n",
    "mask = a>=0.5\n",
    "opened_mask = ndimage.binary_opening(mask)\n",
    "closed_mask = ndimage.binary_closing(opened_mask)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/morpho.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "练习\n",
    "\n",
    "验证一下重建的方格面积比原始方格的面积小。（如果关闭步骤在开启步骤之前则相反）。\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于*灰度值图像*，腐蚀（区别于扩张）相当于用感兴趣的像素周围的结构元素中的最小（区别于最大）值替换像素。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "a = np.zeros((7,7), dtype=np.int)\n",
    "a[1:6, 1:6] = 3\n",
    "a[4,4] = 2; a[2,3] = 1\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "ndimage.grey_erosion(a, size=(3,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.10.4 测量图像\n",
    "\n",
    "首先让我们生成一个漂亮的人造二维图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "x, y = np.indices((100, 100))\n",
    "sig = np.sin(2*np.pi*x/50.)*np.sin(2*np.pi*y/50.)*(1+x*y/50.**2)**2\n",
    "mask = sig > 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在让我们看一下图像中对象的各种信息："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "labels, nb = ndimage.label(mask)\n",
    "nb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "areas = ndimage.sum(mask, labels, range(1, labels.max()+1))\n",
    "areas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "maxima = ndimage.maximum(sig, labels, range(1, labels.max()+1))\n",
    "maxima"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "ndimage.find_objects(labels==4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sl = ndimage.find_objects(labels==4)\n",
    "plt.imshow(sig[sl[0]])  \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/measures.png)\n",
    "\n",
    "高级例子请看一下总结练习[图像处理应用：计数气泡和未融化的颗粒](http://scipy-lectures.github.io/intro/summary-exercises/image-processing.html#summary-exercise-image-processing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5.11 科学计算的总结练习\n",
    "\n",
    "总结练习主要使用Numpy、Scipy 和 Matplotlib。他们提供了一些使用Python进行科学计算的真实例子。现在，已经介绍了Numpy和Scipy的基本使用，邀请感兴趣的用户去做这些练习。\n",
    "\n",
    "**练习：**\n",
    "\n",
    "[1.5.11.1 Sprogø气象站的最大风速预测](http://scipy-lectures.github.io/intro/summary-exercises/stats-interpolate.html)\n",
    "\n",
    "[1.5.11.2 非线性最小二乘曲线拟合：地形机载激光雷达数据中的点抽取](http://scipy-lectures.github.io/intro/summary-exercises/optimize-fit.html)\n",
    "\n",
    "[1.5.11.3 图像处理应用：计数气泡和未融化的颗粒](http://scipy-lectures.github.io/intro/summary-exercises/image-processing.html)\n",
    "\n",
    "**提议的解决方案:**\n",
    "\n",
    "[1.5.11.4 图像处理练习：玻璃中的未融化颗粒的答案例子](http://scipy-lectures.github.io/intro/summary-exercises/answers_image_processing.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.11.1 Sprogø气象站的最大风速预测\n",
    "\n",
    "这个练习的目的是预测每50年的最大风速，即使在一个时间段内有记录。可用的数据只是位于丹麦的Sprogø气象站的21年的测量数据。首先，将给出统计步骤，接着将用scipy.interpolae模块中的函数来解释。在最后，将邀请感兴趣的读者用不同的方法从原始数据计算结果。\n",
    "\n",
    "#### 1.5.11.1.1 统计方法\n",
    "\n",
    "假设年度最大值符合正态概率密度函数。但是，这个函数不能用来预测，因为它从速度最大值中给出了概率。找到每50年的最大风速需要相反的方法，需要从确定的概率中找到结果。这是百分位数函数的作用而这个练习的目的是找到它。在当前的模型中，假设每50年出现的最大风速定义为高于2%百分位数。\n",
    "\n",
    "根据定义，百分位数函数是累积分布函数的反函数。后者描述了年度最大值的概率分布。在这个练习中，给定年份$i$的累积概率$p_i$被定义为$p_i = i/(N+1)$，其中$N = 21$，测量的年数。因此，计算每个测量过的风速最大值的累积概率是可以行的。从这些实验点，scipy.interpolate模块将对拟合百分位数函数非常有用。最后，50年的最大值将从累积概率的2%百分位数中预估出来。\n",
    "\n",
    "#### 1.5.11.1.2 计算累积概率\n",
    "\n",
    "\u001f计算好的numpy格式的年度风速最大值存储在[examples/max-speeds.npy](http://scipy-lectures.github.io/_downloads/max-speeds.npy)文件中,\u001f 因此，可以用numpy加载："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "max_speeds = np.load('data/max-speeds.npy')\n",
    "years_nb = max_speeds.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是前面板块的累积概率定义$p_i$，对应值将为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "cprob = (np.arange(years_nb, dtype=np.float32) + 1)/(years_nb + 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "并且假设他们可以拟合给定的风速："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sorted_max_speeds = np.sort(max_speeds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.5.11.1.3 用UnivariateSpline预测\n",
    "\n",
    "在这个部分，百分位数函数将用`UnivariateSpline`类来估计，这个类用点代表样条。 默认行为是构建一个3度的样条，不同的点根据他们的可靠性可能有不同的权重。相关的变体还有`InterpolatedUnivariateSpline`和`LSQUnivariateSpline`，差别在于检查误差的方式不同。如果需要2D样条，可以使用`BivariateSpline`家族类。所有这些1D和2D样条使用FITPACK Fortran 程序，这就是为什么通过`splrep`和`splev`函数来表征和评估样条的库更少。同时，不使用FITPACK参数的插值函数也提供更简便的用法（见`interp1d`, `interp2d`, `barycentric_interpolate`等等）。\n",
    "对于Sprogø最大风速的例子，将使用`UnivariateSpline`，因为3度的样条似乎可以正确拟合数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy.interpolate import UnivariateSpline\n",
    "quantile_func = UnivariateSpline(cprob, sorted_max_speeds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "百分位数函数将用评估来所有范围的概率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "nprob = np.linspace(0, 1, 1e2)\n",
    "fitted_max_speeds = quantile_func(nprob)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在当前的模型中，每50年出现的最大风速被定义为大于2%百分位数。作为结果，累积概率值将是："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "fifty_prob = 1. - 0.02"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此，可以猜测50年一遇的暴风雪风速为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "fifty_wind = quantile_func(fifty_prob)\n",
    "fifty_wind  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，结果被收集在Matplotlib图片中：\n",
    "\n",
    "![](http://scipy-lectures.github.io/_images/plot_cumulative_wind_speed_prediction_1.png)\n",
    "\n",
    "答案：[Python源文件](http://scipy-lectures.github.io/intro/summary-exercises/auto_examples/plot_cumulative_wind_speed_prediction.html#example-plot-cumulative-wind-speed-prediction-py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.5.11.1.4 Gumbell分布练习\n",
    "\n",
    "现在邀请感兴趣的读者用21年测量的风速做一个练习。测量区间为90分钟（原始的区间约为10分钟，但是，为了让练习的设置简单一些，缩小了文件的大小）。数据以numpy格式存储在文件[examples/sprog-windspeeds.npy](http://scipy-lectures.github.io/_downloads/sprog-windspeeds.npy)中。 在完成练习后，不要看绘图的源代码。\n",
    "\n",
    "- 第一步将是通过使用numpy来找到年度最大值，然后将它们绘制为matplotlibe条形图。\n",
    "![](http://scipy-lectures.github.io/_images/plot_sprog_annual_maxima_1.png)\n",
    "\n",
    "答案：[Python源文件](http://scipy-lectures.github.io/intro/summary-exercises/auto_examples/plot_sprog_annual_maxima.html#example-plot-sprog-annual-maxima-py)\n",
    "\n",
    "- 第二步将是在累积概率$p_i$使用Gumbell分布，$p_i$的定义是$-log( -log(p_i) )$用来拟合线性百分位数函数（记住你可以定义UnivariateSpline的度数）。 绘制年度最大值和Gumbell分布将生产如下图片。\n",
    "\n",
    "![](http://scipy-lectures.github.io/_images/plot_gumbell_wind_speed_prediction_1.png)\n",
    "\n",
    "答案：[Python源文件](http://scipy-lectures.github.io/intro/summary-exercises/auto_examples/plot_gumbell_wind_speed_prediction.html#example-plot-gumbell-wind-speed-prediction-py)\n",
    "\n",
    "- 最后一步将是找到在每50年出现的最大风速34.23 m/s。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.11.2 非线性最小二乘曲线拟合：地理雷达数据中的点抽取应用\n",
    "\n",
    "这个练习的目的是用模型去拟合一些数据。这篇教程中的数据是雷达数据，下面的介绍段落将详细介绍。如果你没有耐心，想要马上进行联系，那么请跳过这部分，并直接进入[加载和可视化](http://scipy-lectures.github.io/intro/summary-exercises/optimize-fit.html#first-step)。\n",
    "\n",
    "#### 1.5.11.2.1 介绍\n",
    "\n",
    "雷达系统是光学测距仪，通过分析离散光的属性来测量距离。绝大多数光学测距仪向目标发射一段短光学脉冲，然后记录反射信号。然后处理这个信号来抽取雷达系统与目标间的距离。\n",
    "\n",
    "地形雷达系统是嵌入在飞行平台的雷达系统。它们测量平台与地球的距离，以便计算出地球的地形信息（更多细节见[[1]](http://scipy-lectures.github.io/intro/summary-exercises/optimize-fit.html#mallet)）。\n",
    "\n",
    "---\n",
    "[1]\tMallet, C. and Bretar, F. Full-Waveform Topographic Lidar: State-of-the-Art. ISPRS Journal of Photogrammetry and Remote Sensing 64(1), pp.1-16, January 2009 http://dx.doi.org/10.1016/j.isprsjprs.2008.09.007\n",
    "\n",
    "---\n",
    "\n",
    "这篇教程的目的是分析雷达系统记录到的波形数据[[2]](http://scipy-lectures.github.io/intro/summary-exercises/optimize-fit.html#data)。这种信号包含波峰，波峰的中心和振幅可以用来计算命中目标的位置和一些特性。当激光柱的脚步距离地球表面1m左右，光柱可以在二次传播时击中多个目标（例如，地面和树木或建筑的顶部）。激光柱的击中每个目标的贡献之和会产生一个有多个波峰的复杂波，每一个包含一个目标的信息。\n",
    "\n",
    "一种从这些数据中抽取信息的先进方法是在一个高斯函数和中分解这些信息，每个函数代表激光柱击中的一个目标的贡献。\n",
    "\n",
    "因此，我们使用`the scipy.optimize`模块将波形拟合为一个高斯函数或高斯函数之和。\n",
    "\n",
    "#### 1.5.11.2.2 加载和可视化\n",
    "\n",
    "加载第一个波形："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "waveform_1 = np.load('data/waveform_1.npy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接着可视化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "t = np.arange(len(waveform_1))\n",
    "plt.plot(t, waveform_1) \n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以注意到，这个波形是单峰80个区间的信息。\n",
    "\n",
    "#### 1.5.11.2.3 用简单的高斯模型拟合波形\n",
    "\n",
    "这个信号非常简单，可以被建模为一个高斯函数，抵消相应的背景噪音。要用函数拟合这个信号，我们必须：\n",
    "- 定义一个模型\n",
    "- 给出初始解\n",
    "- 调用`scipy.optimize.leastsq`\n",
    "\n",
    "##### 1.5.11.2.3.1 模型\n",
    "\n",
    "高斯函数定义如下：\n",
    "\n",
    "$B + A \\exp\\left\\{-\\left(\\frac{t-\\mu}{\\sigma}\\right)^2\\right\\}$\n",
    "\n",
    "在Python中定义如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def model(t, coeffs):\n",
    "    return coeffs[0] + coeffs[1] * np.exp( - ((t-coeffs[2])/coeffs[3])**2 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中\n",
    "\n",
    "- coeffs[0] is $B$ (noise)\n",
    "- coeffs[1] is $A$ (amplitude)\n",
    "- coeffs[2] is $\\mu$ (center)\n",
    "- coeffs[3] is $\\sigma$ (width)\n",
    "\n",
    "##### 1.5.11.2.3.2 初始解\n",
    "\n",
    "通过观察图形，我们可以找到大概的初始解，例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "x0 = np.array([3, 30, 15, 1], dtype=float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.5.11.2.3.3 拟合\n",
    "\n",
    "`scipy.optimize.leastsq`最小化作为参数给到的函数的平方和。本质上来说，函数最小化的是残差（数据与模型的差异）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "def residuals(coeffs, y, t):\n",
    "    return y - model(t, coeffs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此，让我们通过下列参数调用`scipy.optimize.leastsq`来求解：\n",
    "- 最小化的函数\n",
    "- 初始解\n",
    "- 传递给函数的额外参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from scipy.optimize import leastsq\n",
    "x, flag = leastsq(residuals, x0, args=(waveform_1, t))\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "答案可视化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "plt.plot(t, waveform_1, t, model(t, x)) \n",
    "plt.legend(['waveform', 'model']) \n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "备注：从scipy v0.8及以上，你应该使用`scipy.optimize.curve_fit`，它使用模型和数据作为参数，因此，你不再需要定义残差。\n",
    "\n",
    "#### 1.5.11.2.4 更进一步\n",
    "\n",
    "- 试一下包含三个波峰的更复杂波形（例如[data/waveform_2.npy](https://github.com/scipy-lectures/scipy-lecture-notes/raw/master/data/waveform_2.npy)）。你必须调整模型，现在它是高斯函数之和，而不是只有一个高斯波峰。\n",
    "\n",
    "![](http://scipy-lectures.github.io/_images/waveform_2.png)\n",
    "\n",
    "- 在一些情况下，写一个函数来计算Jacobian，要比让leastsq从数值上估计它来的快。创建一个函数来计算残差的Jacobian，并且用它作为leastsq的一个输入。\n",
    "- 当我们想要识别信号中非常小的峰值，或者初始的猜测离好的解决方案太远时，算法给出的结果往往不能令人满意。为模型参数添加限制可以确保克服这些局限性。我们可以添加的先前经验是变量的符号（都是正的）。\n",
    "\n",
    "用下列初始解："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "x0 = np.array([3, 50, 20, 1], dtype=float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "添加了边界限制之后比较一下`scipy.optimize.leastsq`与`scipy.optimize.fmin_slsqp`的结果。\n",
    "\n",
    "---\n",
    "[2]\t本教程的数据部分来自于[FullAnalyze software](http://fullanalyze.sourceforge.net/)的演示数据，由 [GIS DRAIX](http://www.ore.fr/rubrique.php3?id_rubrique=24) 友情提供。\n",
    "\n",
    "---\n",
    "\n",
    "### 1.5.11.3 图像处理应用：计数气泡和未融化的颗粒\n",
    "\n",
    "![](http://scipy-lectures.github.io/_images/MV_HFV_012.jpg)\n",
    "\n",
    "#### 1.5.11.3.1 问题描述\n",
    "\n",
    "1. 打开图像文件MV_HFV_012.jpg并且浏览一下。看一下imshow文档字符串中的参数，用“右”对齐来显示图片（原点在左下角，而不是像标准数组在右上角）。\n",
    "\n",
    "    这个扫描元素显微图显示了一个带有一些气泡（黑色）和未溶解沙（深灰）的玻璃样本（轻灰矩阵）。我们想要判断样本由三个状态覆盖的百分比，并且预测沙粒和气泡的典型大小和他们的大小等。\n",
    "\n",
    "2. 修建图片，删除带有测量信息中底部面板。\n",
    "\n",
    "3. 用中位数过滤稍稍过滤一下图像以便改进它的直方图。看一下直方图的变化。\n",
    "\n",
    "4. 使用过滤后图像的直方图，决定允许定义沙粒像素，玻璃像素和气泡像素掩蔽的阈限。其他的选项（家庭作业）：写一个函数从直方图的最小值自动判断阈限。\n",
    "\n",
    "5. 将三种不同的相用不同的颜色上色并显示图片。\n",
    "\n",
    "6. 用数学形态学清理不同的相。\n",
    "\n",
    "7. 为所有气泡和沙粒做标签，从沙粒中删除小于10像素的掩蔽。要这样做，用`ndimage.sum`或`np.bincount`来计算沙粒大小。\n",
    "\n",
    "8. 计算气泡的平均大小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.5.11.4 图像处理练习：玻璃中的未融化颗粒的答案例子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from scipy import ndimage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/MV_HFV_012.jpg)\n",
    "\n",
    "- 打开图像文件MV_HFV_012.jpg并且浏览一下。看一下imshow文档字符串中的参数，用“右”对齐来显示图片（原点在左下角，而不是像标准数组在右上角）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "dat = plt.imread('data/MV_HFV_012.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 修建图片，删除带有测量信息中底部面板。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "dat = dat[60:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 用中位数过滤稍稍过滤一下图像以便改进它的直方图。看一下直方图的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "filtdat = ndimage.median_filter(dat, size=(7,7))\n",
    "hi_dat = np.histogram(dat, bins=np.arange(256))\n",
    "hi_filtdat = np.histogram(filtdat, bins=np.arange(256))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/exo_histos.png)\n",
    "- 使用过滤后图像的直方图，决定允许定义沙粒像素，玻璃像素和气泡像素掩蔽的阈限。其他的选项（家庭作业）：写一个函数从直方图的最小值自动判断阈限。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "void = filtdat <= 50\n",
    "sand = np.logical_and(filtdat > 50, filtdat <= 114)\n",
    "glass = filtdat > 114"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 将三种不同的相用不同的颜色上色并显示图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "phases = void.astype(np.int) + 2*glass.astype(np.int) + 3*sand.astype(np.int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/three_phases.png)\n",
    "- 用数学形态学清理不同的相。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sand_op = ndimage.binary_opening(sand, iterations=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 为所有气泡和沙粒做标签，从沙粒中删除小于10像素的掩蔽。要这样做，用`ndimage.sum`或`np.bincount`来计算沙粒大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "sand_labels, sand_nb = ndimage.label(sand_op)\n",
    "sand_areas = np.array(ndimage.sum(sand_op, sand_labels, np.arange(sand_labels.max()+1)))\n",
    "mask = sand_areas > 100\n",
    "remove_small_sand = mask[sand_labels.ravel()].reshape(sand_labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](http://scipy-lectures.github.io/_images/sands.png)\n",
    "- 计算气泡的平均大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "bubbles_labels, bubbles_nb = ndimage.label(void)\n",
    "bubbles_areas = np.bincount(bubbles_labels.ravel())[1:]\n",
    "mean_bubble_size = bubbles_areas.mean()\n",
    "median_bubble_size = np.median(bubbles_areas)\n",
    "mean_bubble_size, median_bubble_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}