Biopython_Tutorial_and_Cookbook_Chapter_9.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:4403907d4a14f93e4bc03a8ca5670bb075307c6ebbf7772dc6ab71ee541a0a8f"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Using Biopython's `Bio.Entrez` package to Access NCBI's Entrez databases"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This IPython Notebook is based on Chapter [9](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec118) of the [Biopython Tutorial and Cookbook](http://biopython.org/DIST/docs/tutorial/Tutorial.html), \"Accessing NCBI\u2019s Entrez databases\". \n",
      "\n",
      "Related documents supplied by the [National Center for Biotechnology Information (NCBI)](http://www.ncbi.nlm.nih.gov/) include:\n",
      "\n",
      "* [A General Introduction to the E-utilities](http://www.ncbi.nlm.nih.gov/books/NBK25497)\n",
      "* [The E-utilities In-Depth: Parameters, Syntax and More](http://www.ncbi.nlm.nih.gov/books/NBK25499)\n",
      "\n",
      "The Introduction to the first NCBI document offers the following overview:\n",
      "\n",
      "> The Entrez Programming Utilities (E-utilities) are a set of nine server-side programs that provide a stable interface into the Entrez query and database system at the National Center for Biotechnology Information (NCBI). The E-utilities use a fixed URL syntax that translates a standard set of input parameters into the values necessary for various NCBI software components to search for and retrieve the requested data. The E-utilities are therefore the structured interface to the Entrez system, which currently includes 38 databases covering a variety of biomedical data, including nucleotide and protein sequences, gene records, three-dimensional molecular structures, and the biomedical literature.\n",
      "\n",
      "My primary goal was to learn how to use `Bio.Entrez` to conduct citation analysis on [PubMed](http://www.ncbi.nlm.nih.gov/pubmed) articles. I find the best way to learn how to use nearly any new [Python](http://www.python.org) tool is to create an [IPython Notebook](http://ipython.org/notebook.html) to experiment with sample code and data. I am sharing this notebook in case others find it useful ... and my intention to share the notebook helps motivate me to learn the material better than I might otherwise.\n",
      "\n",
      "I have elaborated on some sections in Chapter [9](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec118) of the [Biopython Tutorial and Cookbook](http://biopython.org/DIST/docs/tutorial/Tutorial.html), inserted links to relevant NCBI and Python documentation, adopted more consistent coding and naming conventions (the Biopython document was developed and revised by different people at different times), corrected some errors (e.g., variable names, file names) and added some error checking (try/except statements) in Section [9.12 (Handling Errors)](#9.11--Handling-errors) so that all the cells in the notebook can be run without interruption.\n",
      "\n",
      "I also addressed a \"TODO\" item in Section [9.15.3](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec143) of the Biopython Tutorial and Cookbook - see the section near the bottom of the notebook, [Update: Searching for citations using ELink, EPost and EFetch with history](#Update:-Searching-for-citations-using-ELink,-EPost-and-EFetch-with-history) below. I encountered an [`HTTPError`](https://docs.python.org/library/urllib2.html#urllib2.HTTPError) while working on an incorrect approach to using ELink and EFetch with the Entrez History server (I initially omitted an EPost call in between those two). Entrez defines an internal `_HTTPError` exception that does not appear to be intended for reuse, but I found it useful to access that, so I added another new section at the bottom, [update on handling HTTPErrors](#Update:-Handling-HTTPErrors).\n",
      "\n",
      "To make the notebook as self-contained as possible, I replaced absolute linked references to sections in Chapter 9 with relative links, and then inserted anchors into the corresponding sections in this notebook (using a technique suggested by [Sebastian Raschka](http://sebastianraschka.com/) in his IPython Notebook, [Creating a table of contents with internal links in IPython Notebooks and Markdown documents](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/table_of_contents_ipython.ipynb)). \n",
      "\n",
      "The [Biopython wiki](http://biopython.org/wiki/Main_Page) has instructions for [downloading and installing Biopython](http://biopython.org/wiki/Download) on various platforms. I use [Anaconda](https://store.continuum.io/cshop/anaconda/) on a Mac, and simply used \n",
      "\n",
      "`conda install biopython` \n",
      "\n",
      "on a Terminal command line for my installation.\n",
      "\n",
      "A final note: when I first tried to run some of the examples in Chapter 9, warning messages indicated that I needed to install the following DTD files, either in a system-wide Python library subfolder (//anaconda/lib/python2.7/site-packages/Bio/Entrez/DTDs) or in a user-specific subfolder (~/.biopython/Bio/Entrez/DTDs). I chose to install them in the system-wide folder:\n",
      "\n",
      "* http://eutils.ncbi.nlm.nih.gov/eutils/dtd/20130322/einfo.dtd\n",
      "* http://eutils.ncbi.nlm.nih.gov/eutils/dtd/20131226/efetch.dtd\n",
      "* http://eutils.ncbi.nlm.nih.gov/eutils/dtd/20060628/esearch.dtd\n",
      "* http://eutils.ncbi.nlm.nih.gov/eutils/dtd/20060131/esummary-v1.dtd\n",
      "* http://www.ncbi.nlm.nih.gov/corehtml/query/DTD/bookdoc_140101.dtd\n",
      "* http://www.ncbi.nlm.nih.gov/corehtml/query/DTD/nlmmedlinecitationset_140101.dtd\n",
      "* http://www.ncbi.nlm.nih.gov/corehtml/query/DTD/pubmed_140101.dtd\n",
      "\n",
      "In a Biopython Github discussion thread on [missing Bio.Entrez DTD files](https://github.com/biopython/biopython/issues/342), I discovered that this was due to Anaconda using Biopython 1.63. [peterjc](https://github.com/peterjc) noted:\n",
      "\n",
      "> Because keeping up to date with the NCBI DTD files is an ongoing task, Biopython 1.64 takes a more automated approach to downloading and caching missing DTD files without making you do it manually. It also should include all of the DTD files you found missing - you can see the current list here https://github.com/biopython/biopython/tree/master/Bio/Entrez/DTDs - if we are still missing any please let us know (we can reopen this issue).\n",
      "\n",
      "So I upgraded to Biopython 1.64 by using the following command in a Terminal window:\n",
      "\n",
      "`conda upgrade biopython`\n",
      "\n",
      "which brought my local biopoython package version up to 1.64, and increased the number of DTD files in my `Bio/Entrez/DTD` directory from 198 to 238."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Biopython Tutorial and Cookbook uses Python 3; it is compatible with Python 2, with the proviso that we adapt the print function\n",
      "from __future__ import print_function"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "[Chapter 9: Accessing NCBI\u2019s Entrez databases](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec118)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[Entrez](http://www.ncbi.nlm.nih.gov/Entrez) is a data retrieval system that provides users access to NCBI\u2019s databases such as PubMed, GenBank, GEO, and many others. You can access Entrez from a web browser to manually enter queries, or you can use [Biopython\u2019s `Bio.Entrez`](http://biopython.org/DIST/docs/api/Bio.Entrez-module.html) module for programmatic access to Entrez. The latter allows you for example to search PubMed or download GenBank records from within a Python script.\n",
      "\n",
      "The `Bio.Entrez` module provides a Python wrapper around the NCBI [Entrez Programming Utilities (EUtils)](http://www.ncbi.nlm.nih.gov/entrez/utils/). Each of these EUtils corresponds to one Python function in the `Bio.Entrez` module, whose name is the same as the corresponding EUtils tool, except it is all lowercase. Among other benefits, this module makes sure that the correct URL is used for the queries, and that not more than one request is made every three seconds, as required by NCBI guidelines.\n",
      "\n",
      "The output returned by the EUtils is typically in XML format. To parse such output, you have several options:\n",
      "\n",
      "1. Use the `Bio.Entrez` parser to parse the XML output into a Python object;\n",
      "2. Use the [DOM (Document Object Model)](https://docs.python.org/2/library/xml.dom.html) parser in Python\u2019s standard library;\n",
      "3. Use the [SAX (Simple API for XML)](https://docs.python.org/2/library/xml.sax.html) parser in Python\u2019s standard library;\n",
      "4. Read the XML output as raw text, and parse it by string searching and manipulation.\n",
      "\n",
      "For the DOM and SAX parsers, see the Python documentation. The parser in `Bio.Entrez` is discussed below.\n",
      "\n",
      "NCBI uses [DTD (Document Type Definition)](http://en.wikipedia.org/wiki/Document_type_definition) files to describe the structure of the information contained in XML files. Most of the [NCBI DTD files](http://www.ncbi.nlm.nih.gov/dtd/) are included in the Biopython distribution. The `Bio.Entrez` parser makes use of these DTD files when parsing an XML file returned by EUtils.\n",
      "\n",
      "Occasionally, you may find that the DTD file associated with a specific XML file is missing in the Biopython distribution. In particular, this may happen when NCBI updates its DTD files. If this happens, `Entrez.read()` will show a warning message with the name and URL of the missing DTD file. The parser will proceed to access the missing DTD file through the internet, allowing the parsing of the XML file to continue. However, the parser is much faster if the DTD file is available locally. \n",
      "\n",
      "To update your local DTD collection, please download the DTD file from the URL in the warning message and place it in your local Python library sudirectory `site-packages/Bio/Entrez/DTDs`, which contains the other DTD files. If you don\u2019t have write access to this directory, you can also place the DTD file in `~/.biopython/Bio/Entrez/DTDs`, where `~` represents your home directory. Since this directory is read before the directory `site-packages/Bio/Entrez/DTDs`, you can also put newer versions of DTD files there if the ones in `site-packages/Bio/Entrez/DTDs` become outdated. Alternatively, if you installed Biopython from source, you can add the DTD file to the source code\u2019s `Bio/Entrez/DTDs` directory, and reinstall Biopython. This will install the new DTD file in the correct location together with the other DTD files.\n",
      "\n",
      "EUtils can also generate output in other formats, such as the Fasta or GenBank file formats for sequence databases, or the MedLine format for the literature database, discussed in Section [9.12](#sec%3Aentrez-specialized-parsers)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aentrez-guidelines\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.1  Entrez Guidelines"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Before using Biopython to access the NCBI\u2019s online resources (via `Bio.Entrez` or some of the other modules), please read the [NCBI Entrez User Requirements](http://www.ncbi.nlm.nih.gov/books/NBK25497/#_chapter2_Usage_Guidelines_and_Requiremen_). If the NCBI finds you are abiding by these guidelines, they can and will block your access!\n",
      "\n",
      "To paraphrase:\n",
      "\n",
      "* For any series of more than 100 requests, do this on weekends or on weekdays outside peak times, i.e., between 9pm and 5am Eastern US time.\n",
      "* Use the http://eutils.ncbi.nlm.nih.gov address, not the standard NCBI Web address. Biopython uses this web address.\n",
      "* Make no more than three requests every second (relaxed from at most one request every three seconds in early 2009). This is automatically enforced by Biopython.\n",
      "* Use the optional email parameter so the NCBI can contact you if there is a problem. You can either explicitly set this as a parameter with each call to Entrez (e.g. include email=\"A.N.Other@example.com\" in the argument list), or you can set a global email address:\n",
      "```\n",
      ">>> from Bio import Entrez\n",
      ">>> Entrez.email = \"A.N.Other@example.com\"\n",
      "```\n",
      "`Bio.Entrez` will then use this email address with each call to Entrez. The `example.com` address used in this tutorial is a reserved domain name specifically for documentation (RFC 2606). Please DO NOT use a random email \u2013 it\u2019s better not to give an email at all. The email parameter will be mandatory from June 1, 2010. In case of excessive usage, NCBI will attempt to contact a user at the e-mail address provided prior to blocking access to the E-utilities.\n",
      "\n",
      "* If you are using Biopython within some larger software suite, use the tool parameter to specify this. You can either explicitly set the tool name as a parameter with each call to Entrez (e.g. include `tool=\"MyLocalScript\"` in the argument list), or you can set a global tool name:\n",
      "```\n",
      ">>> from Bio import Entrez\n",
      ">>> Entrez.tool = \"MyLocalScript\"\n",
      "```\n",
      "The tool parameter will default to `\"Biopython\"`.\n",
      "\n",
      "After importing `Entrez` and setting the module `email` and/or `tool` attributes, working with data retrieved from NCBI EUtils typically involves \n",
      "\n",
      "1. opening up a `handle` - a local file or a file-like object returned by EUtils calls through which data can be retrieved \n",
      "2. reading all the data at once or parsing one data object at a time\n",
      "3. closing the `handle`\n",
      "\n",
      "For large queries, the NCBI recommends using the [Entrez History server](http://www.ncbi.nlm.nih.gov/books/NBK1058/#eutils_esayers-2-3) (which involves WebEnv session cookie string, see Section [9.15](#sec%3Aentrez-webenv)).\n",
      "\n",
      "Be sensible with your usage levels. If you plan to download lots of data, consider other options. For example, if you want easy access to all the human genes, consider fetching each chromosome by FTP as a GenBank file, and importing these into your own BioSQL database (see Section [18.5](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3ABioSQL))."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aentrez-einfo\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.2  EInfo: Obtaining information about the Entrez databases"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[EInfo](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EInfo) provides a list of the names of all valid Entrez databases. If an optional `db` parameter is specified, EInfo provides statistics for that database, including lists of indexing fields and available link names."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.einfo()\n",
      "result = Entrez.read(handle)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The variable `result` now contains a list of databases in XML format:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(result)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{u'DbList': ['pubmed', 'protein', 'nuccore', 'nucleotide', 'nucgss', 'nucest', 'structure', 'genome', 'assembly', 'genomeprj', 'bioproject', 'biosample', 'blastdbinfo', 'books', 'cdd', 'clinvar', 'clone', 'gap', 'gapplus', 'dbvar', 'epigenomics', 'gene', 'gds', 'geoprofiles', 'homologene', 'medgen', 'journals', 'mesh', 'ncbisearch', 'nlmcatalog', 'omim', 'orgtrack', 'pmc', 'popset', 'probe', 'proteinclusters', 'pcassay', 'biosystems', 'pccompound', 'pcsubstance', 'pubmedhealth', 'seqannot', 'snp', 'sra', 'taxonomy', 'toolkit', 'toolkitall', 'toolkitbook', 'unigene', 'gencoll', 'gtr']}\n"
       ]
      }
     ],
     "prompt_number": 3
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Since this is a fairly simple XML file, we could extract the information it contains simply by string searching. Using `Entrez.einfo()`, we can directly parse this XML file into a Python object:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = Entrez.einfo()\n",
      "result = Entrez.read(handle)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now `result` is a dictionary with exactly one key:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "[u'DbList']"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The values stored in this key are the list of database names accessible via Entrez:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"DbList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "['pubmed', 'protein', 'nuccore', 'nucleotide', 'nucgss', 'nucest', 'structure', 'genome', 'assembly', 'genomeprj', 'bioproject', 'biosample', 'blastdbinfo', 'books', 'cdd', 'clinvar', 'clone', 'gap', 'gapplus', 'dbvar', 'epigenomics', 'gene', 'gds', 'geoprofiles', 'homologene', 'medgen', 'journals', 'mesh', 'ncbisearch', 'nlmcatalog', 'omim', 'orgtrack', 'pmc', 'popset', 'probe', 'proteinclusters', 'pcassay', 'biosystems', 'pccompound', 'pcsubstance', 'pubmedhealth', 'seqannot', 'snp', 'sra', 'taxonomy', 'toolkit', 'toolkitall', 'toolkitbook', 'unigene', 'gencoll', 'gtr']"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can use `Entrez.einfo()` to obtain more information about any of these databases. For example, by specifying `db=\"pubmed\"`, we can obtain more information about the `pubmed` database."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = Entrez.einfo(db=\"pubmed\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"DbInfo\"].keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "[u'Count',\n",
        " u'LastUpdate',\n",
        " u'MenuName',\n",
        " u'Description',\n",
        " u'LinkList',\n",
        " u'DbBuild',\n",
        " u'FieldList',\n",
        " u'DbName']"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `\"Count\"` field contains a string representing the number of entries in the database, the `\"LastUpdate\"` field contains a timestamp of the most recent entry.\n",
      "\n",
      "_[Historical note: value returned below was '17989604' on '2008/05/24 06:45']_"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"DbInfo\"][\"Count\"], result[\"DbInfo\"][\"LastUpdate\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "('23953830', '2014/06/26 07:56')"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `\"FieldList\"` field contains a list of possible search fields which can be selected via `Entrez.esearch()`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for field in result[\"DbInfo\"][\"FieldList\"]:\n",
      "    print(\"%(Name)s, %(FullName)s, %(Description)s\" % field)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "ALL, All Fields, All terms from all searchable fields\n",
        "UID, UID, Unique number assigned to publication\n",
        "FILT, Filter, Limits the records\n",
        "TITL, Title, Words in title of publication\n",
        "WORD, Text Word, Free text associated with publication\n",
        "MESH, MeSH Terms, Medical Subject Headings assigned to publication\n",
        "MAJR, MeSH Major Topic, MeSH terms of major importance to publication\n",
        "AUTH, Author, Author(s) of publication\n",
        "JOUR, Journal, Journal abbreviation of publication\n",
        "AFFL, Affiliation, Author's institutional affiliation and address\n",
        "ECNO, EC/RN Number, EC number for enzyme or CAS registry number\n",
        "SUBS, Supplementary Concept, CAS chemical name or MEDLINE Substance Name\n",
        "PDAT, Date - Publication, Date of publication\n",
        "EDAT, Date - Entrez, Date publication first accessible through Entrez\n",
        "VOL, Volume, Volume number of publication\n",
        "PAGE, Pagination, Page number(s) of publication\n",
        "PTYP, Publication Type, Type of publication (e.g., review)\n",
        "LANG, Language, Language of publication\n",
        "ISS, Issue, Issue number of publication\n",
        "SUBH, MeSH Subheading, Additional specificity for MeSH term\n",
        "SI, Secondary Source ID, Cross-reference from publication to other databases\n",
        "MHDA, Date - MeSH, Date publication was indexed with MeSH terms\n",
        "TIAB, Title/Abstract, Free text associated with Abstract/Title\n",
        "OTRM, Other Term, Other terms associated with publication\n",
        "INVR, Investigator, Investigator\n",
        "COLN, Author - Corporate, Corporate Author of publication\n",
        "CNTY, Place of Publication, Country of publication\n",
        "PAPX, Pharmacological Action, MeSH pharmacological action pre-explosions\n",
        "GRNT, Grant Number, NIH Grant Numbers\n",
        "MDAT, Date - Modification, Date of last modification\n",
        "CDAT, Date - Completion, Date of completion\n",
        "PID, Publisher ID, Publisher ID\n",
        "FAUT, Author - First, First Author of publication\n",
        "FULL, Author - Full, Full Author Name(s) of publication\n",
        "FINV, Investigator - Full, Full name of investigator\n",
        "TT, Transliterated Title, Words in transliterated title of publication\n",
        "LAUT, Author - Last, Last Author of publication\n",
        "PPDT, Print Publication Date, Date of print publication\n",
        "EPDT, Electronic Publication Date, Date of Electronic publication\n",
        "LID, Location ID, ELocation ID\n",
        "CRDT, Date - Create, Date publication first accessible through Entrez\n",
        "BOOK, Book, ID of the book that contains the document\n",
        "ED, Editor, Section's Editor\n",
        "ISBN, ISBN, ISBN\n",
        "PUBN, Publisher, Publisher's name\n",
        "AUCL, Author Cluster ID, Author Cluster ID\n",
        "EID, Extended PMID, Extended PMID\n",
        "DSO, DSO, Additional text from the summary\n",
        "AUID, Author - Identifier, Author Identifier\n",
        "PS, Subject - Personal Name, Personal Name as Subject\n"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "That\u2019s a long list, but indirectly this tells you that for the PubMed database, you can do things like `Jones[AUTH]` to search the author field, or `Sanger[AFFL]` to restrict to authors at the Sanger Centre. This can be very handy - especially if you are not so familiar with a particular database."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.3  ESearch: Searching the Entrez databases"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[ESearch](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch) provides a list of UIDs matching a text query in a specified database. The following example retrieves all UIDs in the `\"pubmed\"` datatabase which include the term `\"biopython\"`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"pubmed\", term=\"biopython\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"Count\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "'21'"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "['24929426', '24497503', '24267035', '24194598', '23842806', '23157543', '22909249', '22399473', '21666252', '21210977', '20015970', '19811691', '19773334', '19304878', '18606172', '21585724', '16403221', '16377612', '14871861', '14630660']"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `\"IdList\"` field contains a list of PubMed IDs, including 19304878 which is the PMID for the PubMed entry for a Biopython application note, [Biopython: freely available Python tools for computational molecular biology and bioinformatics](http://www.ncbi.nlm.nih.gov/pubmed/19304878) (_Bioinformatics_. 2009 Jun 1;25(11):1422-3). This and other datatabase entries can then be retrieved using `Entrez.efetch()` by specifying their PubMed IDs (see section [9.6](#sec%3Aefetch)).\n",
      "\n",
      "You can also use ESearch to search [GenBank](http://www.ncbi.nlm.nih.gov/genbank/), the NIH genetic sequence database. Here we\u2019ll do a quick search for the _matK_ gene in _Cypripedioideae_ orchids (see Section [9.2](#sec%3Aentrez-einfo) about EInfo for one way to find out which fields you can search in each Entrez database):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"nucleotide\", term=\"Cypripedioideae[Orgn] AND matK[Gene]\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"Count\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 12,
       "text": [
        "'191'"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "By default, ESearch only includes the first 20 UIDs retrieved in the XML output (you can override the default by using the `retmax` parameter). In this case, although there are 191 database entries associated with our query (`\"Cypripedioideae[Orgn] AND matK[Gene]\"`), only the first 20 records are returned in the `\"IdList\"` field. To access additional results, you need to use the WebEnv history feature \u2013 see Section [9.15](#sec%3Aentrez-webenv)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len(result[\"IdList\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 13,
       "text": [
        "20"
       ]
      }
     ],
     "prompt_number": 13
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "['653816509', '653816507', '578888845', '635173154', '635173152', '635173150', '635173148', '635173146', '402502985', '402502983', '402502981', '402502979', '402502977', '402502975', '402502973', '402502971', '402502969', '402502967', '402502965', '402502963']"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Each of the UIDs is a GenBank identifier. See section [9.6](##sec%3Aefetch) for information on how to use EFetch to retrieve the associated GenBank entries.\n",
      "\n",
      "Note that instead of a species name like `Cypripedioideae[Orgn]`, you can restrict the search using an NCBI taxonomy identifier, which in this case would be `txid158330[Orgn]`. This capability isn\u2019t well documented on the [ESearch help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch), the NCBI explained this feature in reply to an email query. You can often deduce the search term formatting by experimenting with the Entrez web interface. For example, including `complete[prop]` in a genome search restricts the search to just completed genomes.\n",
      "\n",
      "As a final example, let\u2019s get a list of computational journal titles:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"journals\", term=\"computational\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"Count\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 15,
       "text": [
        "'59'"
       ]
      }
     ],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "['39206', '37505', '37511', '37435', '37518', '36366', '39786', '34878', '40350', '30367', '38517', '40761', '33843', '37406', '40084', '36622', '39212', '33823', '32989', '33190']"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The [ESearch help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch) has information about the various options that can be used with this tool."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.4  EPost: Uploading a list of identifiers"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[EPost](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EPost) uploads a list of UIDs to the Entrez History server, or appends a list of UIDs to an existing set of UID lists attached to a Web Environment. The UIDs can be used in subsequent EUtils calls.\n",
      "\n",
      "To give an example of when this is useful, suppose you have a long list of UIDs you want to download using EFetch (e.g., references to sequences or citations). When you make a request with EFetch, your parameters - a list of IDs, the database, etc. - are all turned into a long URL sent to the server. If your list of IDs is long, this URL gets long, and long URLs can break (e.g. some proxies don\u2019t cope well).\n",
      "\n",
      "Instead, you can break this up into two steps, first uploading the list of UIDs using EPost (this uses an \u201cHTML post\u201d internally, rather than an \u201cHTML get\u201d, thus avoiding the long URL problem). With the history support, you can then refer to this long list of UIDs, and retrieve the associated entries with EFetch.\n",
      "\n",
      "Let\u2019s look at a simple example to see how EPost works \u2013 uploading some PubMed identifiers:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "id_list = [\"19304878\", \"18606172\", \"16403221\", \"16377612\", \"14871861\", \"14630660\"]\n",
      "handle = Entrez.epost(db=\"pubmed\", id=\",\".join(id_list))\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 17,
       "text": [
        "{u'QueryKey': '1', u'WebEnv': 'NCID_1_715500721_130.14.22.215_9001_1403790674_901216493'}"
       ]
      }
     ],
     "prompt_number": 17
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `result` of the EPost call includes two elements, `QueryKey` and `WebEnv` which together define your history session. You can save these values in variables `query_key` and `webenv` for use with other EUtils such as EFetch (see Section [9.6](sec%3Aefetch)):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "webenv = result[\"WebEnv\"]\n",
      "query_key = result[\"QueryKey\"]\n",
      "webenv, query_key"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "('NCID_1_715500721_130.14.22.215_9001_1403790674_901216493', '1')"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Section [9.15](#sec%3Aentrez-webenv) shows how to use the history feature."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"#sec%3Aesummary\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.5  ESummary: Retrieving summaries from primary IDs"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[ESummary](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESummary) returns a list of document summary records (DocSums) for database entries associated with a list of input UIDs; it can also be used to return DocSums for a set of UIDs stored on the Entrez History server.\n",
      "\n",
      "We can use ESummary to find out more about the journal with PubMed ID 14630660:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esummary(db=\"pubmed\", id=\"14630660\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 19,
       "text": [
        "[{'Title': 'PDB file parser and structure class implemented in Python.', 'Source': 'Bioinformatics', 'PmcRefCount': 28, 'Issue': '17', 'SO': '2003 Nov 22;19(17):2308-10', 'ISSN': '1367-4803', 'Volume': '19', 'FullJournalName': 'Bioinformatics (Oxford, England)', 'RecordStatus': 'PubMed - indexed for MEDLINE', 'ESSN': '1367-4811', 'ELocationID': '', 'Pages': '2308-10', 'PubStatus': 'ppublish', 'AuthorList': ['Hamelryck T', 'Manderick B'], 'EPubDate': '', 'PubDate': '2003 Nov 22', 'NlmUniqueID': '9808944', 'LastAuthor': 'Manderick B', 'ArticleIds': {'medline': [], 'rid': '14630660', 'pubmed': ['14630660'], 'eid': '14630660'}, u'Item': [], 'History': {'medline': ['2004/07/23 05:00'], 'pubmed': ['2003/11/25 05:00'], 'entrez': '2003/11/25 05:00'}, 'LangList': ['English'], 'HasAbstract': 1, 'References': [], 'PubTypeList': ['Evaluation Studies', 'Journal Article'], u'Id': '14630660'}]"
       ]
      }
     ],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0].keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 20,
       "text": [
        "['Title',\n",
        " 'Source',\n",
        " 'PmcRefCount',\n",
        " 'Issue',\n",
        " 'SO',\n",
        " 'ISSN',\n",
        " 'Volume',\n",
        " 'FullJournalName',\n",
        " 'RecordStatus',\n",
        " 'ESSN',\n",
        " 'ELocationID',\n",
        " 'Pages',\n",
        " 'PubStatus',\n",
        " 'AuthorList',\n",
        " 'EPubDate',\n",
        " 'PubDate',\n",
        " 'NlmUniqueID',\n",
        " 'LastAuthor',\n",
        " 'ArticleIds',\n",
        " u'Item',\n",
        " 'History',\n",
        " 'LangList',\n",
        " 'HasAbstract',\n",
        " 'References',\n",
        " 'PubTypeList',\n",
        " u'Id']"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0][\"Title\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 21,
       "text": [
        "'PDB file parser and structure class implemented in Python.'"
       ]
      }
     ],
     "prompt_number": 21
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aefetch\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.6  EFetch: Downloading full records from Entrez"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[EFetch](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EFetch) returns a list of formatted data records from database entries that are associated with a list of input UIDs; it can also be used to return formatted data records for a set of UIDs stored on the Entrez History server.\n",
      "\n",
      "NCBI support several different file formats for its databases. Requesting a specific, non-default file format from Entrez using EFetch requires specifying `rettype` and/or `retmode` optional arguments. The different combinations are described for each database type in a [table of valid values of `retmode` and `rettype` for EFetch](http://www.ncbi.nlm.nih.gov/books/NBK25499/table/chapter4.chapter4_table1/) in the EUtils documentation.\n",
      "\n",
      "One common usage is downloading sequences in the FASTA or GenBank/GenPept plain text formats (which can then be parsed with `Bio.SeqIO`, see Sections [5.3.1](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3ASeqIO_GenBank_Online) and [9.6](#sec%3Aefetch)). From the _Cypripedioideae_ example above, we can download GenBank record 186972394 using `Bio.Entrez.efetch()`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.efetch(db=\"nucleotide\", id=\"186972394\", rettype=\"gb\", retmode=\"text\")\n",
      "result = handle.read()\n",
      "handle.close()\n",
      "print(result)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "LOCUS       EU490707                1302 bp    DNA     linear   PLN 15-JAN-2009\n",
        "DEFINITION  Selenipedium aequinoctiale maturase K (matK) gene, partial cds;\n",
        "            chloroplast.\n",
        "ACCESSION   EU490707\n",
        "VERSION     EU490707.1  GI:186972394\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Selenipedium aequinoctiale\n",
        "  ORGANISM  Selenipedium aequinoctiale\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; Liliopsida; Asparagales; Orchidaceae;\n",
        "            Cypripedioideae; Selenipedium.\n",
        "REFERENCE   1  (bases 1 to 1302)\n",
        "  AUTHORS   Neubig,K.M., Whitten,W.M., Carlsward,B.S., Blanco,M.A., Endara,L.,\n",
        "            Williams,N.H. and Moore,M.\n",
        "  TITLE     Phylogenetic utility of ycf1 in orchids: a plastid gene more\n",
        "            variable than matK\n",
        "  JOURNAL   Plant Syst. Evol. 277 (1-2), 75-84 (2009)\n",
        "REFERENCE   2  (bases 1 to 1302)\n",
        "  AUTHORS   Neubig,K.M., Whitten,W.M., Carlsward,B.S., Blanco,M.A.,\n",
        "            Endara,C.L., Williams,N.H. and Moore,M.J.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (14-FEB-2008) Department of Botany, University of\n",
        "            Florida, 220 Bartram Hall, Gainesville, FL 32611-8526, USA\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1302\n",
        "                     /organism=\"Selenipedium aequinoctiale\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /specimen_voucher=\"FLAS:Blanco 2475\"\n",
        "                     /db_xref=\"taxon:256374\"\n",
        "     gene            <1..>1302\n",
        "                     /gene=\"matK\"\n",
        "     CDS             <1..>1302\n",
        "                     /gene=\"matK\"\n",
        "                     /codon_start=1\n",
        "                     /transl_table=11\n",
        "                     /product=\"maturase K\"\n",
        "                     /protein_id=\"ACC99456.1\"\n",
        "                     /db_xref=\"GI:186972395\"\n",
        "                     /translation=\"IFYEPVEIFGYDNKSSLVLVKRLITRMYQQNFLISSVNDSNQKG\n",
        "                     FWGHKHFFSSHFSSQMVSEGFGVILEIPFSSQLVSSLEEKKIPKYQNLRSIHSIFPFL\n",
        "                     EDKFLHLNYVSDLLIPHPIHLEILVQILQCRIKDVPSLHLLRLLFHEYHNLNSLITSK\n",
        "                     KFIYAFSKRKKRFLWLLYNSYVYECEYLFQFLRKQSSYLRSTSSGVFLERTHLYVKIE\n",
        "                     HLLVVCCNSFQRILCFLKDPFMHYVRYQGKAILASKGTLILMKKWKFHLVNFWQSYFH\n",
        "                     FWSQPYRIHIKQLSNYSFSFLGYFSSVLENHLVVRNQMLENSFIINLLTKKFDTIAPV\n",
        "                     ISLIGSLSKAQFCTVLGHPISKPIWTDFSDSDILDRFCRICRNLCRYHSGSSKKQVLY\n",
        "                     RIKYILRLSCARTLARKHKSTVRTFMRRLGSGLLEEFFMEEE\"\n",
        "ORIGIN      \n",
        "        1 attttttacg aacctgtgga aatttttggt tatgacaata aatctagttt agtacttgtg\n",
        "       61 aaacgtttaa ttactcgaat gtatcaacag aattttttga tttcttcggt taatgattct\n",
        "      121 aaccaaaaag gattttgggg gcacaagcat tttttttctt ctcatttttc ttctcaaatg\n",
        "      181 gtatcagaag gttttggagt cattctggaa attccattct cgtcgcaatt agtatcttct\n",
        "      241 cttgaagaaa aaaaaatacc aaaatatcag aatttacgat ctattcattc aatatttccc\n",
        "      301 tttttagaag acaaattttt acatttgaat tatgtgtcag atctactaat accccatccc\n",
        "      361 atccatctgg aaatcttggt tcaaatcctt caatgccgga tcaaggatgt tccttctttg\n",
        "      421 catttattgc gattgctttt ccacgaatat cataatttga atagtctcat tacttcaaag\n",
        "      481 aaattcattt acgccttttc aaaaagaaag aaaagattcc tttggttact atataattct\n",
        "      541 tatgtatatg aatgcgaata tctattccag tttcttcgta aacagtcttc ttatttacga\n",
        "      601 tcaacatctt ctggagtctt tcttgagcga acacatttat atgtaaaaat agaacatctt\n",
        "      661 ctagtagtgt gttgtaattc ttttcagagg atcctatgct ttctcaagga tcctttcatg\n",
        "      721 cattatgttc gatatcaagg aaaagcaatt ctggcttcaa agggaactct tattctgatg\n",
        "      781 aagaaatgga aatttcatct tgtgaatttt tggcaatctt attttcactt ttggtctcaa\n",
        "      841 ccgtatagga ttcatataaa gcaattatcc aactattcct tctcttttct ggggtatttt\n",
        "      901 tcaagtgtac tagaaaatca tttggtagta agaaatcaaa tgctagagaa ttcatttata\n",
        "      961 ataaatcttc tgactaagaa attcgatacc atagccccag ttatttctct tattggatca\n",
        "     1021 ttgtcgaaag ctcaattttg tactgtattg ggtcatccta ttagtaaacc gatctggacc\n",
        "     1081 gatttctcgg attctgatat tcttgatcga ttttgccgga tatgtagaaa tctttgtcgt\n",
        "     1141 tatcacagcg gatcctcaaa aaaacaggtt ttgtatcgta taaaatatat acttcgactt\n",
        "     1201 tcgtgtgcta gaactttggc acggaaacat aaaagtacag tacgcacttt tatgcgaaga\n",
        "     1261 ttaggttcgg gattattaga agaattcttt atggaagaag aa\n",
        "//\n",
        "\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The arguments `rettype=\"gb\"` and `retmode=\"text\"` let us download this record in the GenBank format.\n",
      "\n",
      "Note that until Easter 2009, the Entrez EFetch API let you use `\u201cgenbank\u201d` as the return type, however the NCBI now insist on using the official return types of `\u201cgb\u201d` or `\u201cgbwithparts\u201d` (or `\u201cgp\u201d` for proteins) as described on online. Also not that until Feb 2012, the Entrez EFetch API would default to returning plain text files, but now defaults to XML.\n",
      "\n",
      "Alternatively, you could for example use `rettype=\"fasta\"` to get the Fasta-format; see the [EFetch Sequences Help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EFetch) for other options. Remember \u2013 the available formats depend on which database you are downloading from - see the main [EFetch Help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EFetch).\n",
      "\n",
      "If you fetch the record in one of the formats accepted by Bio.SeqIO (see Chapter 5), you could directly parse it into a SeqRecord:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez, SeqIO\n",
      "handle = Entrez.efetch(db=\"nucleotide\", id=\"186972394\", rettype=\"gb\", retmode=\"text\")\n",
      "record = SeqIO.read(handle, \"genbank\")\n",
      "handle.close()\n",
      "print(record)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "ID: EU490707.1\n",
        "Name: EU490707\n",
        "Description: Selenipedium aequinoctiale maturase K (matK) gene, partial cds; chloroplast.\n",
        "Number of features: 3\n",
        "/sequence_version=1\n",
        "/source=chloroplast Selenipedium aequinoctiale\n",
        "/taxonomy=['Eukaryota', 'Viridiplantae', 'Streptophyta', 'Embryophyta', 'Tracheophyta', 'Spermatophyta', 'Magnoliophyta', 'Liliopsida', 'Asparagales', 'Orchidaceae', 'Cypripedioideae', 'Selenipedium']\n",
        "/keywords=['']\n",
        "/references=[Reference(title='Phylogenetic utility of ycf1 in orchids: a plastid gene more variable than matK', ...), Reference(title='Direct Submission', ...)]\n",
        "/accessions=['EU490707']\n",
        "/data_file_division=PLN\n",
        "/date=15-JAN-2009\n",
        "/organism=Selenipedium aequinoctiale\n",
        "/gi=186972394\n",
        "Seq('ATTTTTTACGAACCTGTGGAAATTTTTGGTTATGACAATAAATCTAGTTTAGTA...GAA', IUPACAmbiguousDNA())\n"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that a more typical use would be to save the sequence data to a local file, and then parse it with [`Bio.SeqIO`](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec17). This can save you having to re-download the same file repeatedly while working on your script, and places less load on the NCBI servers. For example:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import os\n",
      "from Bio import SeqIO\n",
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "filename = \"gi_186972394.gbk\"\n",
      "if not os.path.isfile(filename): # don't overwrite file\n",
      "    # Downloading a single entry\n",
      "    net_handle = Entrez.efetch(db=\"nucleotide\", id=\"186972394\", rettype=\"gb\", retmode=\"text\")\n",
      "    net_result = net_handle.read() # using read() rather than SeqIO.read()\n",
      "    net_handle.close()\n",
      "    with open(filename, \"w\") as out_handle:\n",
      "        out_handle.write(net_result)\n",
      "    print(\"Saved\")\n",
      "\n",
      "print(\"Parsing...\")\n",
      "result = SeqIO.read(filename, \"genbank\")\n",
      "print(result)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Parsing...\n",
        "ID: EU490707.1\n",
        "Name: EU490707\n",
        "Description: Selenipedium aequinoctiale maturase K (matK) gene, partial cds; chloroplast.\n",
        "Number of features: 3\n",
        "/sequence_version=1\n",
        "/source=chloroplast Selenipedium aequinoctiale\n",
        "/taxonomy=['Eukaryota', 'Viridiplantae', 'Streptophyta', 'Embryophyta', 'Tracheophyta', 'Spermatophyta', 'Magnoliophyta', 'Liliopsida', 'Asparagales', 'Orchidaceae', 'Cypripedioideae', 'Selenipedium']\n",
        "/keywords=['']\n",
        "/references=[Reference(title='Phylogenetic utility of ycf1 in orchids: a plastid gene more variable than matK', ...), Reference(title='Direct Submission', ...)]\n",
        "/accessions=['EU490707']\n",
        "/data_file_division=PLN\n",
        "/date=15-JAN-2009\n",
        "/organism=Selenipedium aequinoctiale\n",
        "/gi=186972394\n",
        "Seq('ATTTTTTACGAACCTGTGGAAATTTTTGGTTATGACAATAAATCTAGTTTAGTA...GAA', IUPACAmbiguousDNA())\n"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To get the output in XML format, which you can parse into a list of dictionaries using the `Entrez.read()` function, use `retmode=\"xml\"`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = Entrez.efetch(db=\"nucleotide\", id=\"186972394\", retmode=\"xml\")\n",
      "record = Entrez.read(handle)\n",
      "handle.close()\n",
      "record[0][\"GBSeq_definition\"] "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "'Selenipedium aequinoctiale maturase K (matK) gene, partial cds; chloroplast'"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "record[0][\"GBSeq_source\"] "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        "'chloroplast Selenipedium aequinoctiale'"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, that dealt with sequences. For examples of parsing file formats specific to the other databases (e.g. the MEDLINE format used in PubMed), see Section [9.12](#sec%3Aentrez-specialized-parsers).\n",
      "\n",
      "If you want to perform a search with ESearch, and then download the records with EFetch, you should use the WebEnv history feature \u2013 see Section [9.15](#sec%3Aentrez-webenv)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aelink\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.7  ELink: Searching for related items in NCBI Entrez"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[ELink](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ELink) returns UIDs linked to an input set of UIDs in either the same or a different Entrez database, and can be used to find related items in the NCBI Entrez databases. For example, you can us this to find nucleotide entries for an entry in the gene database, and other cool stuff.\n",
      "\n",
      "Let\u2019s use ELink to find articles related to the Biopython application note published in _Bioinformatics_ in 2009. The PubMed ID of this article is 19304878:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"\n",
      "pmid = \"19304878\"\n",
      "handle = Entrez.elink(dbfrom=\"pubmed\", id=pmid)\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "len(result), result[0].keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "(1, [u'LinkSetDb', u'DbFrom', u'IdList', u'LinkSetDbHistory', u'ERROR'])"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `result` variable consists of a Python list, one for each database in which we searched. Since we specified only one PubMed ID to search for, `result` contains only one item. This item is a dictionary containing information about our search term, as well as all the related items that were found:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0][\"DbFrom\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 28,
       "text": [
        "'pubmed'"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0][\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 29,
       "text": [
        "['19304878']"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `\"LinkSetDb\"` key contains the search results, stored as a list consisting of one item for each target database. In our search results, we only find hits in the PubMed database (although sub-divided into categories):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len(result[0][\"LinkSetDb\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 30,
       "text": [
        "8"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for linksetdb in result[0][\"LinkSetDb\"]:\n",
      "    print(linksetdb[\"DbTo\"], linksetdb[\"LinkName\"], len(linksetdb[\"Link\"]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "pubmed pubmed_pubmed 177\n",
        "pubmed pubmed_pubmed_alsoviewed 1\n",
        "pubmed pubmed_pubmed_citedin 143\n",
        "pubmed pubmed_pubmed_combined 6\n",
        "pubmed pubmed_pubmed_five 6\n",
        "pubmed pubmed_pubmed_refs 17\n",
        "pubmed pubmed_pubmed_reviews 8\n",
        "pubmed pubmed_pubmed_reviews_five 6\n"
       ]
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The actual search results are stored as under the `\"Link\"` key. The total number of links found is the value of the entry `\"pubmed_pubmed\"`. Let\u2019s now at the first search result:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0][\"LinkSetDb\"][0][\"Link\"][0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 32,
       "text": [
        "{u'Id': '19304878'}"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This is the article we searched for, which doesn\u2019t help us much, so let\u2019s look at the second search result:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[0][\"LinkSetDb\"][0][\"Link\"][1]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 33,
       "text": [
        "{u'Id': '14630660'}"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This paper, with PubMed ID 14630660, is about the Biopython PDB parser.\n",
      "\n",
      "We can use a loop to print out all PubMed IDs:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for link in result[0][\"LinkSetDb\"][0][\"Link\"]:\n",
      "    print(link[\"Id\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "19304878\n",
        "14630660\n",
        "22909249\n",
        "20739307\n",
        "23023984\n",
        "18689808\n",
        "20733063\n",
        "19628504\n",
        "18251993\n",
        "15572471\n",
        "20847218\n",
        "20823319\n",
        "12368254\n",
        "23842806\n",
        "20421198\n",
        "22399473\n",
        "21216774\n",
        "22581176\n",
        "22302572\n",
        "18238804\n",
        "24064416\n",
        "22332238\n",
        "15723693\n",
        "15096277\n",
        "14512356\n",
        "17237069\n",
        "16377612\n",
        "15130828\n",
        "18593718\n",
        "20973958\n",
        "12230038\n",
        "22368248\n",
        "23633579\n",
        "20591906\n",
        "21352538\n",
        "22815363\n",
        "21949271\n",
        "17101041\n",
        "19181685\n",
        "16257987\n",
        "17441614\n",
        "18227118\n",
        "17316423\n",
        "17384428\n",
        "16539535\n",
        "22824207\n",
        "22877863\n",
        "14751976\n",
        "16899494\n",
        "17237072\n",
        "23479348\n",
        "22500002\n",
        "16569235\n",
        "20537149\n",
        "20375454\n",
        "19106120\n",
        "20334363\n",
        "20439314\n",
        "19460889\n",
        "21385461\n",
        "17537750\n",
        "15969769\n",
        "23456039\n",
        "15059834\n",
        "23292976\n",
        "14871861\n",
        "15383216\n",
        "22276101\n",
        "19698094\n",
        "17483505\n",
        "19773334\n",
        "17291351\n",
        "15260898\n",
        "20472540\n",
        "17586821\n",
        "16741236\n",
        "17121776\n",
        "20007251\n",
        "18442177\n",
        "23493324\n",
        "21798964\n",
        "16371163\n",
        "19958528\n",
        "22788675\n",
        "17586553\n",
        "20022974\n",
        "16188925\n",
        "21210984\n",
        "17496317\n",
        "21210978\n",
        "16922600\n",
        "20542914\n",
        "21737439\n",
        "22084254\n",
        "12401134\n",
        "22942023\n",
        "11524374\n",
        "17496320\n",
        "19336443\n",
        "17483515\n",
        "22556367\n",
        "16796559\n",
        "16539540\n",
        "22396485\n",
        "22253821\n",
        "22595207\n",
        "23071651\n",
        "23846743\n",
        "22039207\n",
        "15980476\n",
        "22942017\n",
        "23666736\n",
        "21715385\n",
        "19578173\n",
        "23699471\n",
        "22908215\n",
        "23355290\n",
        "21685053\n",
        "24885957\n",
        "22565567\n",
        "22813356\n",
        "23742983\n",
        "23956303\n",
        "24078714\n",
        "22494792\n",
        "23516352\n",
        "23418185\n",
        "23220574\n",
        "23367449\n",
        "23422340\n",
        "19648141\n",
        "15461798\n",
        "23809062\n",
        "24068901\n",
        "24600386\n",
        "23957210\n",
        "24431986\n",
        "23242262\n",
        "22024252\n",
        "24420968\n",
        "23987304\n",
        "22171336\n",
        "23348786\n",
        "21210977\n",
        "23435069\n",
        "24045775\n",
        "22238272\n",
        "23574738\n",
        "23628689\n",
        "21984743\n",
        "21458441\n",
        "22936991\n",
        "22844241\n",
        "24618462\n",
        "21362187\n",
        "23786315\n",
        "23765606\n",
        "24511080\n",
        "24795618\n",
        "22645166\n",
        "22954632\n",
        "22369160\n",
        "21803787\n",
        "23633576\n",
        "19225577\n",
        "16729046\n",
        "20924230\n",
        "23652425\n",
        "24258321\n",
        "21716279\n",
        "24929426\n",
        "22479120\n",
        "24834575\n",
        "24267035\n",
        "21253560\n",
        "24359023\n",
        "24945496\n"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now that was nice, but personally I am often more interested to find out if a paper has been cited. Well, ELink can do that too \u2013 at least for journals in Pubmed Central (see Section [9.15.3](#sec%3Aelink-citations)).\n",
      "\n",
      "For help on ELink, see the [ELink help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ELink). There is an entire sub-page just for the [link names](http://eutils.ncbi.nlm.nih.gov/entrez/query/static/entrezlinks.html), describing how different databases can be cross referenced."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.8  EGQuery: Global Query - counts for search terms"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[EGQuery](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EGQuery) provides counts for a search term in each of the Entrez databases (i.e. a global query). This is particularly useful to find out how many items your search terms would find in each database without actually performing lots of separate searches with ESearch (see the example in [9.14.2](#subsec%3Aentrez_example_genbank) below).\n",
      "\n",
      "In this example, we use EGQuery to obtain the number of articles associated the term \u201cbiopython\u201d:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.egquery(term=\"biopython\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "for row in result[\"eGQueryResult\"]:\n",
      "    print(row[\"DbName\"], row[\"Count\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "pubmed 21\n",
        "pmc 371\n",
        "mesh 0\n",
        "books 2\n",
        "pubmedhealth 0\n",
        "omim 0\n",
        "ncbisearch 0\n",
        "nuccore 0\n",
        "nucgss 0\n",
        "nucest 0\n",
        "protein 0\n",
        "genome 0\n",
        "structure 0\n",
        "taxonomy 0\n",
        "snp 0\n",
        "dbvar 0\n",
        "epigenomics 0\n",
        "gene 0\n",
        "sra 0\n",
        "biosystems 0\n",
        "unigene 0\n",
        "cdd 0\n",
        "clone 0\n",
        "popset 0\n",
        "geoprofiles 0\n",
        "gds 16\n",
        "homologene 0\n",
        "pccompound 0\n",
        "pcsubstance 0\n",
        "pcassay 0\n",
        "nlmcatalog 0\n",
        "probe 0\n",
        "gap 0\n",
        "proteinclusters 0\n",
        "bioproject 0\n",
        "biosample 0\n"
       ]
      }
     ],
     "prompt_number": 35
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "See the [EGQuery help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EGQuery) for more information. "
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.9  ESpell: Obtaining spelling suggestions"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[ESpell](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESpell) retrieves spelling suggestions. In this example, we use ESpell to obtain the correct spelling of \"Biopython\":"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.espell(term=\"biopythooon\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result.keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 36,
       "text": [
        "[u'Query', u'SpelledQuery', u'CorrectedQuery', u'Database']"
       ]
      }
     ],
     "prompt_number": 36
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"Query\"], result[\"CorrectedQuery\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 37,
       "text": [
        "('biopythooon', 'biopython')"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "See the [ESpell help page](http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESpell) for more information. The main use of this is for GUI tools to provide automatic suggestions for search terms."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.10  Parsing huge Entrez XML files"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `Entrez.read()` function reads the entire XML file returned by Entrez into a single Python object, which is kept in memory. To parse Entrez XML files too large to fit in memory, you can use the function `Entrez.parse()`. This is a [generator function](https://docs.python.org/tutorial/classes.html#generators) that reads records in the XML file one by one. This function is only useful if the XML file represents a Python list object (in other words, if `Entrez.read()` on a computer with infinite memory resources would return a Python list). The handle passed to `Entrez.parse()` should not be closed until all elements have been parsed.\n",
      "\n",
      "For example, you can download the entire Entrez Gene database for a given organism as a file from NCBI\u2019s ftp site. These files can be very large. As an example, on September 4, 2009, the file `Homo_sapiens.ags.gz`, containing the Entrez Gene database for human, had a size of 116576 kB. This file, which is in the [`ASN`](http://www.ncbi.nlm.nih.gov/Structure/asn1.html) format, can be converted into an XML file using NCBI\u2019s `gene2xml` program (see [NCBI\u2019s ftp site](http://www.filewatcher.com/b/ftp/ftp.ncbi.nlm.nih.gov/entrez/toolbox/ncbi_tools/converters/by_program/gene2xml-0.html) for more information):\n",
      "\n",
      "`gene2xml -b T -i Homo_sapiens.ags -o Homo_sapiens.xml`\n",
      "\n",
      "The resulting XML file has a size of 6.1 GB. Attempting `Entrez.read()` on this file will result in a [`MemoryError`](https://docs.python.org/library/exceptions.html#exceptions.MemoryError) on many computers.\n",
      "\n",
      "The XML file `Homo_sapiens.xml` consists of a list of Entrez gene records, each corresponding to one Entrez gene in human. `Entrez.parse()` retrieves these gene records one by one. You can then print out or store the relevant information in each record by iterating over the records. For example, this script iterates over the Entrez gene records and prints out the gene numbers and names for all current genes:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = open(\"Homo_sapiens.xml\")\n",
      "result = Entrez.parse(handle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for i, record in enumerate(result): # using enumerate to enable breaking after a specified number of iterations\n",
      "    if i > 10: # print at most 10 records\n",
      "        break\n",
      "    status = record['Entrezgene_track-info']['Gene-track']['Gene-track_status']\n",
      "    if status.attributes['value'] == 'discontinued':\n",
      "        continue\n",
      "    geneid = record['Entrezgene_track-info']['Gene-track']['Gene-track_geneid']\n",
      "    genename = record['Entrezgene_gene']['Gene-ref']['Gene-ref_locus']\n",
      "    print(geneid, genename)\n",
      "handle.close() # note: do not close the handle until done with parsing"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "1 A1BG\n",
        "2"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " A2M\n",
        "3"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " A2MP1\n",
        "8 AA\n",
        "9"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " NAT1\n",
        "10"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " NAT2\n",
        "11"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        " NATP\n"
       ]
      }
     ],
     "prompt_number": 39
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.11  Handling errors"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Three things can go wrong when parsing an XML file:\n",
      "\n",
      "* The file may not be an XML file to begin with;\n",
      "* The file may end prematurely or otherwise be corrupted;\n",
      "* The file may be correct XML, but contain items that are not represented in the associated DTD.\n",
      "\n",
      "The first case occurs if, for example, you try to parse a Fasta file as if it were an XML file (after downloading [NC_005816.fna](http://biopython.org/SRC/biopython/Tests/GenBank/NC_005816.fna)):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = open(\"NC_005816.fna\") # a Fasta file\n",
      "try:\n",
      "    result = Entrez.read(handle)\n",
      "except Entrez.Parser.NotXMLError as e:\n",
      "    print(e)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Failed to parse the XML data (syntax error: line 1, column 0). Please make sure that the input data are in XML format.\n"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here, the parser didn\u2019t find the `<?xml ...` tag with which an XML file is supposed to start, and therefore decides (correctly) that the file is not an XML file.\n",
      "\n",
      "When your file is in the XML format but is corrupted (for example, by ending prematurely), the parser will raise a `CorruptedXMLError` Here is an example of an XML file that ends prematurely:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "with open('corruptedxmlerror_example.xml', 'w') as f:\n",
      "    f.write('''<?xml version=\"1.0\"?>\n",
      "<!DOCTYPE eInfoResult PUBLIC \"-//NLM//DTD eInfoResult, 11 May 2002//EN\" \"http://www.ncbi.nlm.nih.gov/entrez/query/DTD/eInfo_020511.dtd\">\n",
      "<eInfoResult>\n",
      "<DbList>\n",
      "        <DbName>pubmed</DbName>\n",
      "        <DbName>protein</DbName>\n",
      "        <DbName>nucleotide</DbName>\n",
      "        <DbName>nuccore</DbName>\n",
      "        <DbName>nucgss</DbName>\n",
      "        <DbName>nucest</DbName>\n",
      "        <DbName>structure</DbName>\n",
      "        <DbName>genome</DbName>\n",
      "        <DbName>books</DbName>\n",
      "        <DbName>cancerchromosomes</DbName>\n",
      "        <DbName>cdd</DbName>\n",
      "        ''')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 41
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "which will generate an `Entrez.Parser.CorruptedXMLError` exception during the attempt to parse it:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = open('corruptedxmlerror_example.xml')\n",
      "try:\n",
      "    result = Entrez.read(handle)\n",
      "except Entrez.Parser.CorruptedXMLError as e:\n",
      "    print(e)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Failed to parse the XML data (no element found: line 16, column 8). Please make sure that the input data are not corrupted.\n"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that the error message tells you at what point in the XML file the error was detected.\n",
      "\n",
      "The third type of error occurs if the XML file contains tags that do not have a description in the corresponding DTD file. This is an example of such an XML file:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "with open('einfo3.xml', 'w') as f:\n",
      "    f.write('''<?xml version=\"1.0\"?>\n",
      "<!DOCTYPE eInfoResult PUBLIC \"-//NLM//DTD eInfoResult, 11 May 2002//EN\" \"http://www.ncbi.nlm.nih.gov/entrez/query/DTD/eInfo_020511.dtd\">\n",
      "<eInfoResult>\n",
      "    <DbInfo>\n",
      "        <DbName>pubmed</DbName>\n",
      "        <MenuName>PubMed</MenuName>\n",
      "        <Description>PubMed bibliographic record</Description>\n",
      "        <Count>20161961</Count>\n",
      "        <LastUpdate>2010/09/10 04:52</LastUpdate>\n",
      "        <FieldList>\n",
      "            <Field>\n",
      "            ...\n",
      "            </Field>\n",
      "        </FieldList>\n",
      "        <DocsumList>\n",
      "            <Docsum>\n",
      "                <DsName>PubDate</DsName>\n",
      "                <DsType>4</DsType>\n",
      "                <DsTypeName>string</DsTypeName>\n",
      "            </Docsum>\n",
      "            <Docsum>\n",
      "                <DsName>EPubDate</DsName>\n",
      "                ...\n",
      "            </Docsum>\n",
      "        </DocsumList>\n",
      "    </DbInfo>\n",
      "</eInfoResult>\n",
      "''')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 43
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In this file, for some reason the tag `<DocsumList>` (and several others) are not listed in the DTD file `eInfo_020511.dtd`, which is specified on the second line as the DTD for this XML file. By default, the parser will stop and raise a `ValidationError` if it cannot find some tag in the DTD:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = open(\"einfo3.xml\")\n",
      "try:\n",
      "    result = Entrez.read(handle)\n",
      "except Entrez.Parser.ValidationError as e:\n",
      "    print(e)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Failed to find tag 'DocsumList' in the DTD. To skip all tags that are not represented in the DTD, please call Bio.Entrez.read or Bio.Entrez.parse with validate=False.\n"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Optionally, you can instruct the parser to skip such tags instead of raising a `ValidationError`. This is done by calling `Entrez.read()` or `Entrez.parse()` with the option `validate=False`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = open(\"einfo3.xml\")\n",
      "result = Entrez.read(handle, validate=False)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Of course, the information contained in the XML tags that are not defined in the DTD are not included in the record returned by `Entrez.read()`."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aentrez-specialized-parsers\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.12  Specialized parsers"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `Entrez.read()` function can parse most (if not all) XML output returned by EUtils. Entrez typically allows you to retrieve records in other formats, which may have some advantages compared to the XML format in terms of readability (or download size).\n",
      "\n",
      "To request a specific file format from Entrez using EFetch requires specifying the `rettype` and/or `retmode` optional arguments. The different combinations are described for each database type in a [table of valid values of `retmode` and `rettype` for EFetch](http://www.ncbi.nlm.nih.gov/books/NBK25499/table/chapter4.chapter4_table1/) in the EUtils documentation\n",
      "\n",
      "One obvious case is you may prefer to download sequences in the FASTA or GenBank/GenPept plain text formats (which can then be parsed with `Bio.SeqIO`, see Sections [5.3.1](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3ASeqIO_GenBank_Online) and [9.6](#sec%3Aefetch)). For the literature databases, Biopython contains a parser for the MEDLINE format used in PubMed."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"subsec%3Aentrez-and-medline\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.12.1  Parsing Medline records"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can use the Medline parser in `Bio.Medline` using calls to `Medline.read()`. Suppose we want to parse the file `pubmed_result1.txt`, containing one Medline record. (This file can be foundin Biopython\u2019s [`Tests\\Medline` directory](https://github.com/biopython/biopython/tree/master/Tests/Medline)). The file looks like this:\n",
      "\n",
      "```\n",
      "PMID- 12230038\n",
      "OWN - NLM\n",
      "STAT- MEDLINE\n",
      "DA  - 20020916\n",
      "DCOM- 20030606\n",
      "LR  - 20041117\n",
      "PUBM- Print\n",
      "IS  - 1467-5463 (Print)\n",
      "VI  - 3\n",
      "IP  - 3\n",
      "DP  - 2002 Sep\n",
      "TI  - The Bio* toolkits--a brief overview.\n",
      "PG  - 296-302\n",
      "AB  - Bioinformatics research is often difficult to do with commercial software. The\n",
      "      Open Source BioPerl, BioPython and Biojava projects provide toolkits with\n",
      "...\n",
      "```"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We first open the file and then parse it:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Medline\n",
      "with open(\"pubmed_result1.txt\") as handle:\n",
      "    result = Medline.read(handle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `record` variable now contains the Medline record as a Python dictionary:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"PMID\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "'12230038'"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"AB\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 48,
       "text": [
        "'Bioinformatics research is often difficult to do with commercial software. The Open Source BioPerl, BioPython and Biojava projects provide toolkits with multiple functionality that make it easier to create customised pipelines or analysis. This review briefly compares the quirks of the underlying languages and the functionality, documentation, utility and relative advantages of the Bio counterparts, particularly from the point of view of the beginning biologist programmer.'"
       ]
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The key names used in a Medline record can be rather obscure; use `help` for a brief summary."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "help(result)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Help on Record in module Bio.Medline object:\n",
        "\n",
        "class Record(__builtin__.dict)\n",
        " |  A dictionary holding information from a Medline record.\n",
        " |  All data are stored under the mnemonic appearing in the Medline\n",
        " |  file. These mnemonics have the following interpretations:\n",
        " |  \n",
        " |  Mnemonic  Description\n",
        " |  AB        Abstract\n",
        " |  CI        Copyright Information\n",
        " |  AD        Affiliation\n",
        " |  IRAD      Investigator Affiliation\n",
        " |  AID       Article Identifier\n",
        " |  AU        Author\n",
        " |  FAU       Full Author\n",
        " |  CN        Corporate Author\n",
        " |  DCOM      Date Completed\n",
        " |  DA        Date Created\n",
        " |  LR        Date Last Revised\n",
        " |  DEP       Date of Electronic Publication\n",
        " |  DP        Date of Publication\n",
        " |  EDAT      Entrez Date\n",
        " |  GS        Gene Symbol\n",
        " |  GN        General Note\n",
        " |  GR        Grant Number\n",
        " |  IR        Investigator Name\n",
        " |  FIR       Full Investigator Name\n",
        " |  IS        ISSN\n",
        " |  IP        Issue\n",
        " |  TA        Journal Title Abbreviation\n",
        " |  JT        Journal Title\n",
        " |  LA        Language\n",
        " |  LID       Location Identifier\n",
        " |  MID       Manuscript Identifier\n",
        " |  MHDA      MeSH Date\n",
        " |  MH        MeSH Terms\n",
        " |  JID       NLM Unique ID\n",
        " |  RF        Number of References\n",
        " |  OAB       Other Abstract\n",
        " |  OCI       Other Copyright Information\n",
        " |  OID       Other ID\n",
        " |  OT        Other Term\n",
        " |  OTO       Other Term Owner\n",
        " |  OWN       Owner\n",
        " |  PG        Pagination\n",
        " |  PS        Personal Name as Subject\n",
        " |  FPS       Full Personal Name as Subject\n",
        " |  PL        Place of Publication\n",
        " |  PHST      Publication History Status\n",
        " |  PST       Publication Status\n",
        " |  PT        Publication Type\n",
        " |  PUBM      Publishing Model\n",
        " |  PMC       PubMed Central Identifier\n",
        " |  PMID      PubMed Unique Identifier\n",
        " |  RN        Registry Number/EC Number\n",
        " |  NM        Substance Name\n",
        " |  SI        Secondary Source ID\n",
        " |  SO        Source\n",
        " |  SFM       Space Flight Mission\n",
        " |  STAT      Status\n",
        " |  SB        Subset\n",
        " |  TI        Title\n",
        " |  TT        Transliterated Title\n",
        " |  VI        Volume\n",
        " |  CON       Comment on\n",
        " |  CIN       Comment in\n",
        " |  EIN       Erratum in\n",
        " |  EFR       Erratum for\n",
        " |  CRI       Corrected and Republished in\n",
        " |  CRF       Corrected and Republished from\n",
        " |  PRIN      Partial retraction in\n",
        " |  PROF      Partial retraction of\n",
        " |  RPI       Republished in\n",
        " |  RPF       Republished from\n",
        " |  RIN       Retraction in\n",
        " |  ROF       Retraction of\n",
        " |  UIN       Update in\n",
        " |  UOF       Update of\n",
        " |  SPIN      Summary for patients in\n",
        " |  ORI       Original report in\n",
        " |  \n",
        " |  Method resolution order:\n",
        " |      Record\n",
        " |      __builtin__.dict\n",
        " |      __builtin__.object\n",
        " |  \n",
        " |  Data descriptors defined here:\n",
        " |  \n",
        " |  __dict__\n",
        " |      dictionary for instance variables (if defined)\n",
        " |  \n",
        " |  __weakref__\n",
        " |      list of weak references to the object (if defined)\n",
        " |  \n",
        " |  ----------------------------------------------------------------------\n",
        " |  Methods inherited from __builtin__.dict:\n",
        " |  \n",
        " |  __cmp__(...)\n",
        " |      x.__cmp__(y) <==> cmp(x,y)\n",
        " |  \n",
        " |  __contains__(...)\n",
        " |      D.__contains__(k) -> True if D has a key k, else False\n",
        " |  \n",
        " |  __delitem__(...)\n",
        " |      x.__delitem__(y) <==> del x[y]\n",
        " |  \n",
        " |  __eq__(...)\n",
        " |      x.__eq__(y) <==> x==y\n",
        " |  \n",
        " |  __ge__(...)\n",
        " |      x.__ge__(y) <==> x>=y\n",
        " |  \n",
        " |  __getattribute__(...)\n",
        " |      x.__getattribute__('name') <==> x.name\n",
        " |  \n",
        " |  __getitem__(...)\n",
        " |      x.__getitem__(y) <==> x[y]\n",
        " |  \n",
        " |  __gt__(...)\n",
        " |      x.__gt__(y) <==> x>y\n",
        " |  \n",
        " |  __init__(...)\n",
        " |      x.__init__(...) initializes x; see help(type(x)) for signature\n",
        " |  \n",
        " |  __iter__(...)\n",
        " |      x.__iter__() <==> iter(x)\n",
        " |  \n",
        " |  __le__(...)\n",
        " |      x.__le__(y) <==> x<=y\n",
        " |  \n",
        " |  __len__(...)\n",
        " |      x.__len__() <==> len(x)\n",
        " |  \n",
        " |  __lt__(...)\n",
        " |      x.__lt__(y) <==> x<y\n",
        " |  \n",
        " |  __ne__(...)\n",
        " |      x.__ne__(y) <==> x!=y\n",
        " |  \n",
        " |  __repr__(...)\n",
        " |      x.__repr__() <==> repr(x)\n",
        " |  \n",
        " |  __setitem__(...)\n",
        " |      x.__setitem__(i, y) <==> x[i]=y\n",
        " |  \n",
        " |  __sizeof__(...)\n",
        " |      D.__sizeof__() -> size of D in memory, in bytes\n",
        " |  \n",
        " |  clear(...)\n",
        " |      D.clear() -> None.  Remove all items from D.\n",
        " |  \n",
        " |  copy(...)\n",
        " |      D.copy() -> a shallow copy of D\n",
        " |  \n",
        " |  fromkeys(...)\n",
        " |      dict.fromkeys(S[,v]) -> New dict with keys from S and values equal to v.\n",
        " |      v defaults to None.\n",
        " |  \n",
        " |  get(...)\n",
        " |      D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.\n",
        " |  \n",
        " |  has_key(...)\n",
        " |      D.has_key(k) -> True if D has a key k, else False\n",
        " |  \n",
        " |  items(...)\n",
        " |      D.items() -> list of D's (key, value) pairs, as 2-tuples\n",
        " |  \n",
        " |  iteritems(...)\n",
        " |      D.iteritems() -> an iterator over the (key, value) items of D\n",
        " |  \n",
        " |  iterkeys(...)\n",
        " |      D.iterkeys() -> an iterator over the keys of D\n",
        " |  \n",
        " |  itervalues(...)\n",
        " |      D.itervalues() -> an iterator over the values of D\n",
        " |  \n",
        " |  keys(...)\n",
        " |      D.keys() -> list of D's keys\n",
        " |  \n",
        " |  pop(...)\n",
        " |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.\n",
        " |      If key is not found, d is returned if given, otherwise KeyError is raised\n",
        " |  \n",
        " |  popitem(...)\n",
        " |      D.popitem() -> (k, v), remove and return some (key, value) pair as a\n",
        " |      2-tuple; but raise KeyError if D is empty.\n",
        " |  \n",
        " |  setdefault(...)\n",
        " |      D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D\n",
        " |  \n",
        " |  update(...)\n",
        " |      D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.\n",
        " |      If E present and has a .keys() method, does:     for k in E: D[k] = E[k]\n",
        " |      If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v\n",
        " |      In either case, this is followed by: for k in F: D[k] = F[k]\n",
        " |  \n",
        " |  values(...)\n",
        " |      D.values() -> list of D's values\n",
        " |  \n",
        " |  viewitems(...)\n",
        " |      D.viewitems() -> a set-like object providing a view on D's items\n",
        " |  \n",
        " |  viewkeys(...)\n",
        " |      D.viewkeys() -> a set-like object providing a view on D's keys\n",
        " |  \n",
        " |  viewvalues(...)\n",
        " |      D.viewvalues() -> an object providing a view on D's values\n",
        " |  \n",
        " |  ----------------------------------------------------------------------\n",
        " |  Data and other attributes inherited from __builtin__.dict:\n",
        " |  \n",
        " |  __hash__ = None\n",
        " |  \n",
        " |  __new__ = <built-in method __new__ of type object>\n",
        " |      T.__new__(S, ...) -> a new object with type S, a subtype of T\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To parse a file containing multiple Medline records, you can use the `Medline.parse` function instead (the `pubmed_result2.txt` file can also be found in Biopython\u2019s [`Tests\\Medline` directory](https://github.com/biopython/biopython/tree/master/Tests/Medline)):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Medline\n",
      "with open(\"pubmed_result2.txt\") as handle:\n",
      "    for result in Medline.parse(handle):\n",
      "        print(result[\"TI\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "A high level interface to SCOP and ASTRAL implemented in python.\n",
        "GenomeDiagram: a python package for the visualization of large-scale genomic data.\n",
        "Open source clustering software.\n",
        "PDB file parser and structure class implemented in Python.\n"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Instead of parsing Medline records stored in files, you can also parse Medline records downloaded by EFetch. For example, let\u2019s use ESearch to find all Medline records in PubMed that use the term `\"biopython\"`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"pubmed\", term=\"biopython\")\n",
      "result = Entrez.read(handle) # Edit: s/record/response/, as record binding was changed in for loop above\n",
      "handle.close()\n",
      "result[\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "['24929426', '24497503', '24267035', '24194598', '23842806', '23157543', '22909249', '22399473', '21666252', '21210977', '20015970', '19811691', '19773334', '19304878', '18606172', '21585724', '16403221', '16377612', '14871861', '14630660']"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We now use EFetch to download these Medline records:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "idlist = result[\"IdList\"]\n",
      "handle2 = Entrez.efetch(db=\"pubmed\", id=idlist, rettype=\"medline\", retmode=\"text\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 52
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here, we specify `rettype=\"medline\", retmode=\"text\"` to obtain the Medline article records in plain-text Medline format. Now we use `Medline.parse` to parse these article records:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Medline\n",
      "result2 = Medline.parse(handle2)\n",
      "for record in result2:\n",
      "    print(record[\"AU\"])\n",
      "handle2.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['Waldmann J', 'Gerken J', 'Hankeln W', 'Schweer T', 'Glockner FO']\n",
        "['Mielke CJ', 'Mandarino LJ', 'Dinu V']\n",
        "['Gajda MJ']\n",
        "['Mathelier A', 'Zhao X', 'Zhang AW', 'Parcy F', 'Worsley-Hunt R', 'Arenillas DJ', 'Buchman S', 'Chen CY', 'Chou A', 'Ienasescu H', 'Lim J', 'Shyr C', 'Tan G', 'Zhou M', 'Lenhard B', 'Sandelin A', 'Wasserman WW']"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "['Morales HF', 'Giovambattista G']\n",
        "['Baldwin S', 'Revanna R', 'Thomson S', 'Pither-Joyce M', 'Wright K', 'Crowhurst R', 'Fiers M', 'Chen L', 'Macknight R', 'McCallum JA']\n",
        "['Talevich E', 'Invergo BM', 'Cock PJ', 'Chapman BA']\n",
        "['Prins P', 'Goto N', 'Yates A', 'Gautier L', 'Willis S', 'Fields C', 'Katayama T']\n",
        "['Schmitt T', 'Messina DN', 'Schreiber F', 'Sonnhammer EL']"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "['Antao T']\n",
        "['Cock PJ', 'Fields CJ', 'Goto N', 'Heuer ML', 'Rice PM']\n",
        "['Jankun-Kelly TJ', 'Lindeman AD', 'Bridges SM']\n",
        "['Korhonen J', 'Martinmaki P', 'Pizzi C', 'Rastas P', 'Ukkonen E']\n",
        "['Cock PJ', 'Antao T', 'Chang JT', 'Chapman BA', 'Cox CJ', 'Dalke A', 'Friedberg I', 'Hamelryck T', 'Kauff F', 'Wilczynski B', 'de Hoon MJ']\n",
        "['Munteanu CR', 'Gonzalez-Diaz H', 'Magalhaes AL']\n",
        "['Faircloth BC']\n",
        "['Casbon JA', 'Crooks GE', 'Saqi MA']\n",
        "['Pritchard L', 'White JA', 'Birch PR', 'Toth IK']\n",
        "['de Hoon MJ', 'Imoto S', 'Nolan J', 'Miyano S']"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "['Hamelryck T', 'Manderick B']\n"
       ]
      }
     ],
     "prompt_number": 53
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For comparison, here we show an example using the XML format (`retmode=\"xml\"`):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "idlist = result[\"IdList\"]\n",
      "handle3 = Entrez.efetch(db=\"pubmed\", id=idlist, rettype=\"medline\", retmode=\"xml\")\n",
      "result3 = Entrez.read(handle3)\n",
      "handle3.close()\n",
      "for record in result3:\n",
      "    print(record[\"MedlineCitation\"][\"Article\"][\"ArticleTitle\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "FastaValidator: an open-source Java library to parse and validate FASTA formatted sequences.\n",
        "AMASS: a database for investigating protein structures.\n",
        "HPDB - Haskell library for processing atomic biomolecular structures in Protein Data Bank format.\n",
        "JASPAR 2014: an extensively expanded and updated open-access database of transcription factor binding profiles.\n",
        "BioSmalltalk: a pure object system and library for bioinformatics.\n",
        "A toolkit for bulk PCR-based marker design from next-generation sequence data: application for development of a framework linkage map in bulb onion (Allium cepa L.).\n",
        "Bio.Phylo: a unified toolkit for processing, analyzing and visualizing phylogenetic trees in Biopython.\n",
        "Sharing programming resources between Bio* projects through remote procedure call and native call stack strategies.\n",
        "Letter to the editor: SeqXML and OrthoXML: standards for sequence and orthology information.\n",
        "interPopula: a Python API to access the HapMap Project dataset.\n",
        "The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants.\n",
        "Exploratory visual analysis of conserved domains on multiple sequence alignments.\n",
        "MOODS: fast search for position weight matrix matches in DNA sequences.\n",
        "Biopython: freely available Python tools for computational molecular biology and bioinformatics.\n",
        "Enzymes/non-enzymes classification model complexity based on composition, sequence, 3D and topological indices.\n",
        "msatcommander: detection of microsatellite repeat arrays and automated, locus-specific primer design.\n",
        "A high level interface to SCOP and ASTRAL implemented in python.\n",
        "GenomeDiagram: a python package for the visualization of large-scale genomic data.\n",
        "Open source clustering software.\n",
        "PDB file parser and structure class implemented in Python.\n"
       ]
      }
     ],
     "prompt_number": 54
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that in both of these examples, for simplicity we have naively combined ESearch and EFetch. In this situation, the NCBI would expect you to use their history feature, as illustrated in Section [9.15](#sec%3Aentrez-webenv)."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.12.2  Parsing GEO records"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "GEO ([Gene Expression Omnibus](http://www.ncbi.nlm.nih.gov/geo/)) is a data repository of high-throughput gene expression and hybridization array data. The `Bio.Geo` module can be used to parse GEO-formatted data.\n",
      "\n",
      "The following code fragment shows how to parse the example GEO file `GSE16.txt` (which can be found in Biopython's [`Tests\\Geo`](https://github.com/biopython/biopython/tree/master/Tests/Geo) directory) into a record and print the record:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Geo\n",
      "handle = open(\"GSE16.txt\")\n",
      "result = Geo.parse(handle)\n",
      "for record in result:\n",
      "     print(record)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "GEO Type: SAMPLE\n",
        "GEO Id: GSM804\n",
        "Sample_author: Antoine,M,Snijders\n",
        "\n",
        "Sample_author: Norma,,Nowak\n",
        "\n",
        "Sample_author: Richard,,Segraves\n",
        "\n",
        "Sample_author: Stephanie,,Blackwood\n",
        "\n",
        "Sample_author: Nils,,Brown\n",
        "\n",
        "Sample_author: Jeffery,,Conroy\n",
        "\n",
        "Sample_author: Greg,,Hamilton\n",
        "\n",
        "Sample_author: Anna,K,Hindle\n",
        "\n",
        "Sample_author: Bing,,Huey\n",
        "\n",
        "Sample_author: Karen,,Kimura\n",
        "\n",
        "Sample_author: Sindy,,Law\n",
        "\n",
        "Sample_author: Ken,,Myambo\n",
        "\n",
        "Sample_author: Joel,,Palmer\n",
        "\n",
        "Sample_author: Bauke,,Ylstra\n",
        "\n",
        "Sample_author: Jingzhu,P,Yue\n",
        "\n",
        "Sample_author: Joe,W,Gray\n",
        "\n",
        "Sample_author: Ajay,N,Jain\n",
        "\n",
        "Sample_author: Daniel,,Pinkel\n",
        "\n",
        "Sample_author: Donna,G,Albertson\n",
        "\n",
        "Sample_description: Coriell Cell Repositories cell line <a h\n",
        "ref=\"http://locus.umdnj.edu/nigms/nigms_cgi/display.cgi?GM05296\">GM05296</a>.\n",
        "\n",
        "Sample_description: Fibroblast cell line derived from a 1 mo\n",
        "nth old female with multiple congenital malformations, dysmorphic features, intr\n",
        "auterine growth retardation, heart murmur, cleft palate, equinovarus deformity, \n",
        "microcephaly, coloboma of right iris, clinodactyly, reduced RBC catalase activit\n",
        "y, and 1 copy of catalase gene.\n",
        "\n",
        "Sample_description: Chromosome abnormalities are present.\n",
        "\n",
        "Sample_description: Karyotype is 46,XX,-11,+der(11)inv ins(1\n",
        "1;10)(11pter> 11p13::10q21>10q24::11p13>11qter)mat\n",
        "\n",
        "Sample_organism: Homo sapiens\n",
        "\n",
        "Sample_platform_id: GPL28\n",
        "\n",
        "Sample_pubmed_id: 11687795\n",
        "\n",
        "Sample_series_id: GSE16\n",
        "\n",
        "Sample_status: Public on Feb 12 2002\n",
        "\n",
        "Sample_submission_date: Jan 17 2002\n",
        "\n",
        "Sample_submitter_city: San Francisco,CA,94143,USA\n",
        "\n",
        "Sample_submitter_department: Comprehensive Cancer Center\n",
        "\n",
        "Sample_submitter_email: albertson@cc.ucsf.edu\n",
        "\n",
        "Sample_submitter_institute: University of California San Francisco\n",
        "\n",
        "Sample_submitter_name: Donna,G,Albertson\n",
        "\n",
        "Sample_submitter_phone: 415 502-8463\n",
        "\n",
        "Sample_target_source1: Cell line GM05296\n",
        "\n",
        "Sample_target_source2: normal male reference genomic DNA\n",
        "\n",
        "Sample_title: CGH_Albertson_GM05296-001218\n",
        "\n",
        "Sample_type: dual channel genomic\n",
        "\n",
        "Column Header Definitions\n",
        "    ID_REF: Unique row identifier, genome position o\n",
        "    rder\n",
        "\n",
        "    LINEAR_RATIO: Mean of replicate Cy3/Cy5 ratios\n",
        "\n",
        "    LOG2STDDEV: Standard deviation of VALUE\n",
        "\n",
        "    NO_REPLICATES: Number of replicate spot measurements\n",
        "\n",
        "    VALUE: aka LOG2RATIO, mean of log base 2 of LIN\n",
        "    EAR_RATIO\n",
        "\n",
        "0: ID_REF\tVALUE\tLINEAR_RATIO\tLOG2STDDEV\tNO_REPLICATES\t\n",
        "1: 1\t\t1.047765\t0.011853\t3\t\n",
        "2: 2\t\t\t\t0\t\n",
        "3: 3\t0.008824\t1.006135\t0.00143\t3\t\n",
        "4: 4\t-0.000894\t0.99938\t0.001454\t3\t\n",
        "5: 5\t0.075875\t1.054\t0.003077\t3\t\n",
        "6: 6\t0.017303\t1.012066\t0.005876\t2\t\n",
        "7: 7\t-0.006766\t0.995321\t0.013881\t3\t\n",
        "8: 8\t0.020755\t1.014491\t0.005506\t3\t\n",
        "9: 9\t-0.094938\t0.936313\t0.012662\t3\t\n",
        "10: 10\t-0.054527\t0.96291\t0.01073\t3\t\n",
        "11: 11\t-0.025057\t0.982782\t0.003855\t3\t\n",
        "12: 12\t\t\t\t0\t\n",
        "13: 13\t0.108454\t1.078072\t0.005196\t3\t\n",
        "14: 14\t0.078633\t1.056017\t0.009165\t3\t\n",
        "15: 15\t0.098571\t1.070712\t0.007834\t3\t\n",
        "16: 16\t0.044048\t1.031003\t0.013651\t3\t\n",
        "17: 17\t0.018039\t1.012582\t0.005471\t3\t\n",
        "18: 18\t-0.088807\t0.9403\t0.010571\t3\t\n",
        "19: 19\t0.016349\t1.011397\t0.007113\t3\t\n",
        "20: 20\t0.030977\t1.021704\t0.016798\t3\t\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You can search the `\"gds\"` database (GEO datasets) with ESearch:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\" # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"gds\", term=\"GSE16\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"Count\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 56,
       "text": [
        "'27'"
       ]
      }
     ],
     "prompt_number": 56
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "len(result[\"IdList\"]), result[\"IdList\"] # note: result[\"IdList\"] will have at most 20 UIDs, since we did not specify retmax"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 57,
       "text": [
        "(20,\n",
        " ['200000016', '100000028', '300000818', '300000817', '300000816', '300000815', '300000814', '300000813', '300000812', '300000811', '300000810', '300000809', '300000808', '300000807', '300000806', '300000805', '300000804', '300000803', '300000802', '300000801'])"
       ]
      }
     ],
     "prompt_number": 57
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "From the Entrez website, UID \u201c200000016\u201d is GDS16 while the other hit \u201c100000028\u201d is for the associated platform, GPL28. Unfortunately, at the time of writing the NCBI don\u2019t seem to support downloading GEO files using Entrez (not as XML, nor in the _Simple Omnibus Format in Text (SOFT)_ format).\n",
      "\n",
      "However, it is actually pretty straight forward to download the GEO files by FTP from ftp://ftp.ncbi.nih.gov/pub/geo/ instead. In this case you might want ftp://ftp.ncbi.nih.gov/pub/geo/DATA/SOFT/by_series/GSE16/GSE16_family.soft.gz (a compressed file, see the Python module [`gzip`](https://docs.python.org/library/gzip.html))."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.12.3  Parsing UniGene records"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "[UniGene](http://www.ncbi.nlm.nih.gov/unigene) is an NCBI database of the transcriptome, with each UniGene record showing the set of transcripts that are associated with a particular gene in a specific organism. A typical UniGene record looks like this:\n",
      "\n",
      "```\n",
      "ID          Hs.2\n",
      "TITLE       N-acetyltransferase 2 (arylamine N-acetyltransferase)\n",
      "GENE        NAT2\n",
      "CYTOBAND    8p22\n",
      "GENE_ID     10\n",
      "LOCUSLINK   10\n",
      "HOMOL       YES\n",
      "EXPRESS      bone| connective tissue| intestine| liver| liver tumor| normal| soft tissue/muscle tissue tumor| adult\n",
      "RESTR_EXPR   adult\n",
      "CHROMOSOME  8\n",
      "STS         ACC=PMC310725P3 UNISTS=272646\n",
      "STS         ACC=WIAF-2120 UNISTS=44576\n",
      "STS         ACC=G59899 UNISTS=137181\n",
      "...\n",
      "STS         ACC=GDB:187676 UNISTS=155563\n",
      "PROTSIM     ORG=10090; PROTGI=6754794; PROTID=NP_035004.1; PCT=76.55; ALN=288\n",
      "PROTSIM     ORG=9796; PROTGI=149742490; PROTID=XP_001487907.1; PCT=79.66; ALN=288\n",
      "PROTSIM     ORG=9986; PROTGI=126722851; PROTID=NP_001075655.1; PCT=76.90; ALN=288\n",
      "...\n",
      "PROTSIM     ORG=9598; PROTGI=114619004; PROTID=XP_519631.2; PCT=98.28; ALN=288\n",
      "\n",
      "SCOUNT      38\n",
      "SEQUENCE    ACC=BC067218.1; NID=g45501306; PID=g45501307; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=NM_000015.2; NID=g116295259; PID=g116295260; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=D90042.1; NID=g219415; PID=g219416; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=D90040.1; NID=g219411; PID=g219412; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=BC015878.1; NID=g16198419; PID=g16198420; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=CR407631.1; NID=g47115198; PID=g47115199; SEQTYPE=mRNA\n",
      "SEQUENCE    ACC=BG569293.1; NID=g13576946; CLONE=IMAGE:4722596; END=5'; LID=6989; SEQTYPE=EST; TRACE=44157214\n",
      "...\n",
      "SEQUENCE    ACC=AU099534.1; NID=g13550663; CLONE=HSI08034; END=5'; LID=8800; SEQTYPE=EST\n",
      "//\n",
      "```\n",
      "\n",
      "This particular record shows the set of transcripts (shown in the SEQUENCE lines) that originate from the human gene [NAT2](http://www.ncbi.nlm.nih.gov/gene/10), encoding en N-acetyltransferase. The `PROTSIM` lines show proteins with significant similarity to NAT2, whereas the `STS` lines show the corresponding sequence-tagged sites in the genome.\n",
      "\n",
      "To parse UniGene files, use the `Bio.UniGene` module (the file `Hs.2.data` can be found in Biopython's [`Tests\\UniGene`](https://github.com/biopython/biopython/tree/master/Tests/UniGene) directory):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import UniGene\n",
      "handle = open(\"Hs.2.data\") # Edit: s/input/handle/, s/myunigene.data/Hs.2.data/\n",
      "result = UniGene.read(handle)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 58
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that `UniGene.read()` returns a Python object with attributes corresponding to the fields in the UniGene record, rather than the dictionary-like object returned by `Entrez.read()`."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.ID"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 59,
       "text": [
        "'Hs.2'"
       ]
      }
     ],
     "prompt_number": 59
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.title"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 60,
       "text": [
        "'N-acetyltransferase 2 (arylamine N-acetyltransferase)'"
       ]
      }
     ],
     "prompt_number": 60
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The `EXPRESS` and `RESTR_EXPR` lines are stored as Python lists of strings:\n",
      "\n",
      "`['bone', 'connective tissue', 'intestine', 'liver', 'liver tumor', 'normal', 'soft tissue/muscle tissue tumor', 'adult']`\n",
      "\n",
      "Specialized objects are returned for the `STS`, `PROTSIM`, and `SEQUENCE` lines, storing the keys shown in each line as attributes:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.sts[0].acc"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 61,
       "text": [
        "'PMC310725P3'"
       ]
      }
     ],
     "prompt_number": 61
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.sts[0].unists"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 62,
       "text": [
        "'272646'"
       ]
      }
     ],
     "prompt_number": 62
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "and similarly for the `PROTSIM` and `SEQUENCE` lines.\n",
      "\n",
      "To parse a file containing more than one UniGene record, use the `UniGene.parse()`:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import UniGene\n",
      "handle = open(\"Eca.1.2425.data\") # Edit: s/input/handle/, s/unigenerecords.data/Eca.1.2425.data/\n",
      "result = UniGene.parse(handle)\n",
      "for record in result:\n",
      "    print(record.ID, record.title)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Eca.1 Ribosomal protein L3\n",
        "Eca.2425 Immunoglobulin-like transcript 11 protein\n"
       ]
      }
     ],
     "prompt_number": 63
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.13  Using a proxy"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Normally you won\u2019t have to worry about using a proxy, but if this is an issue on your network here is how to deal with it. Internally, `Bio.Entrez` uses the standard Python library [`urllib`](https://docs.python.org/library/urllib.html) for accessing the NCBI servers. This will check an environment variable called `http_proxy` to configure any simple proxy automatically. Unfortunately this module does not support the use of proxies which require authentication.\n",
      "\n",
      "You may choose to set the `http_proxy` environment variable once (how you do this will depend on your operating system).\n",
      "\n",
      "Alternatively you can set this within Python at the start of your script, for example:\n",
      "\n",
      "```\n",
      "import os\n",
      "os.environ[\"http_proxy\"] = \"http://proxyhost.example.com:8080\"\n",
      "```\n",
      "\n",
      "See the [`urllib`](https://docs.python.org/library/urllib.html) documentation for more details."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.14  Examples"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.14.1  PubMed and Medline"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If you are in the medical field or interested in human issues (and many times even if you are not!), [PubMed] (http://www.ncbi.nlm.nih.gov/PubMed/) is an excellent source of all kinds of goodies. So like other things, we\u2019d like to be able to retrieve information from it and use it in Python scripts.\n",
      "\n",
      "In this example, we will use EGQuery to search PubMed for all articles having to do with orchids (see section [2.3](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3Aorchids) for our motivation). We first check how many of such articles there are:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.egquery(term=\"orchid\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "print(\"# of databases containing entries for '%s': %i\" % (result['Term'], len(result[\"eGQueryResult\"])))\n",
      "for row in result[\"eGQueryResult\"]:\n",
      "    if row[\"DbName\"] == \"pubmed\":\n",
      "        print(\"# of entries in pubmed database containing '%s': %s\" % (result['Term'], row[\"Count\"]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "# of databases containing entries for 'orchid': 36\n",
        "# of entries in pubmed database containing 'orchid': 1134\n"
       ]
      }
     ],
     "prompt_number": 64
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we use ESearch to retrieve the PubMed IDs of some of these articles (NCBI recommends limiting the number of UIDs to 500 when [handling large data sets](http://www.ncbi.nlm.nih.gov/books/NBK1058/#_large-datasets_))"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "handle = Entrez.esearch(db=\"pubmed\", term=\"orchid\", retmax=500)\n",
      "result = Entrez.read(handle)\n",
      "idlist = result[\"IdList\"]\n",
      "idlist[:10] # show just the first 10"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 65,
       "text": [
        "['24962394',\n",
        " '24961116',\n",
        " '24959588',\n",
        " '24952917',\n",
        " '24916060',\n",
        " '24913627',\n",
        " '24911363',\n",
        " '24907253',\n",
        " '24889383',\n",
        " '24845051']"
       ]
      }
     ],
     "prompt_number": 65
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now that we\u2019ve got a list of PubMed IDs, we can retrieve the corresponding Medline records and extract the information from them. Here, we\u2019ll use EFetch to retrieve the Medline records in the Medline flat-file format, and use the `Bio.Medline` module to parse them:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Medline\n",
      "handle = Entrez.efetch(db=\"pubmed\", id=idlist, rettype=\"medline\", retmode=\"text\")\n",
      "result = Medline.parse(handle)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 66
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "NOTE - We\u2019ve just done a separate search and fetch here, the NCBI much prefer you to take advantage of their history support in this situation. See Section [9.15](#sec%3Aentrez-webenv).\n",
      "\n",
      "Since `parse()` methods in all Biopython modules return [generator objects](https://docs.python.org/tutorial/classes.html#generators), you can iterate through the individual items only once. If you want to save the items for repeated processing, you can convert them to a list:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result = list(result)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 67
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let\u2019s now iterate over the first 10 individual items to print out some information about each record:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for i, record in enumerate(result):\n",
      "    if i > 10:\n",
      "        break\n",
      "    print(\"title:\", record.get(\"TI\", \"?\"))\n",
      "    print(\"authors:\", record.get(\"AU\", \"?\"))\n",
      "    print(\"source:\", record.get(\"SO\", \"?\"))\n",
      "    print(\"\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "title: Volatile fingerprint of italian populations of orchids using solid phase microextraction and gas chromatography coupled with mass spectrometry.\n",
        "authors: ['Manzo A', 'Panseri S', 'Vagge I', 'Giorgi A']\n",
        "source: Molecules. 2014 Jun 11;19(6):7913-36. doi: 10.3390/molecules19067913.\n",
        "\n",
        "title: [Molecular characterization of a HMG-CoA reductase gene from a rare and endangered medicinal plant, Dendrobium officinale].\n",
        "authors: ['Zhang L', 'Wang JT', 'Zhang DW', 'Zhang G', 'Guo SX']\n",
        "source: Yao Xue Xue Bao. 2014 Mar;49(3):411-8.\n",
        "\n",
        "title: Antitumor Activity of Ethanolic Extract of Dendrobium formosum in T-Cell Lymphoma: An In Vitro and In Vivo Study.\n",
        "authors: ['Prasad R', 'Koch B']\n",
        "source: Biomed Res Int. 2014;2014:753451. doi: 10.1155/2014/753451. Epub 2014 May 18.\n",
        "\n",
        "title: New insight into the regulation of floral morphogenesis.\n",
        "authors: ['Tsai WC', 'Pan ZJ', 'Su YY', 'Liu ZJ']\n",
        "source: Int Rev Cell Mol Biol. 2014;311:157-82. doi: 10.1016/B978-0-12-800179-0.00003-9.\n",
        "\n",
        "title: Effects of Pollination Limitation and Seed Predation on Female Reproductive Success of a Deceptive Orchid.\n",
        "authors: ['Walsh RP', 'Arnold PM', 'Michaels HJ']\n",
        "source: AoB Plants. 2014 Jun 9. pii: plu031.\n",
        "\n",
        "title: Multiple isoforms of phosphoenolpyruvate carboxylase in the Orchidaceae (subtribe Oncidiinae): implications for the evolution of crassulacean acid metabolism.\n",
        "authors: ['Silvera K', 'Winter K', 'Rodriguez BL', 'Albion RL', 'Cushman JC']\n",
        "source: J Exp Bot. 2014 Jun 9. pii: eru234.\n",
        "\n",
        "title: Comparative chloroplast genomes of photosynthetic orchids: insights into evolution of the orchidaceae and development of molecular markers for phylogenetic applications.\n",
        "authors: ['Luo J', 'Hou BW', 'Niu ZT', 'Liu W', 'Xue QY', 'Ding XY']\n",
        "source: PLoS One. 2014 Jun 9;9(6):e99016. doi: 10.1371/journal.pone.0099016. eCollection 2014.\n",
        "\n",
        "title: Speciation via floral heterochrony and presumed mycorrhizal host switching of endemic butterfly orchids on the Azorean archipelago.\n",
        "authors: ['Bateman RM', 'Rudall PJ', 'Bidartondo MI', 'Cozzolino S', 'Tranchida-Lombardo V', 'Carine MA', 'Moura M']\n",
        "source: Am J Bot. 2014 Jun 6;101(6):979-1001.\n",
        "\n",
        "title: Pollen competition between two sympatric Orchis species (Orchidaceae): the overtaking of conspecific of heterospecific pollen as a reproductive barrier.\n",
        "authors: ['Luca A', 'Palermo AM', 'Bellusci F', 'Pellegrino G']\n",
        "source: Plant Biol (Stuttg). 2014 May 30. doi: 10.1111/plb.12199.\n",
        "\n",
        "title: In vitro conservation of Dendrobium germplasm.\n",
        "authors: ['Teixeira da Silva JA', 'Zeng S', 'Galdiano RF Jr', 'Dobranszki J', 'Cardoso JC', 'Vendrame WA']\n",
        "source: Plant Cell Rep. 2014 May 21.\n",
        "\n",
        "title: Gigantol, a Bibenzyl from Dendrobium draconis, Inhibits the Migratory Behavior of Non-Small Cell Lung Cancer Cells.\n",
        "authors: ['Charoenrungruang S', 'Chanvorachote P', 'Sritularak B', 'Pongrakhananon V']\n",
        "source: J Nat Prod. 2014 May 20.\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 68
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Note that the list of authors is returned as a standard Python list. This makes it easy to manipulate and search using standard Python tools.\n",
      "\n",
      "For example, we could use a [Counter](https://docs.python.org/library/collections.html#counter-objects) (a specialized Python dictionary) to count the number of articles associated with each author of an orchid-related article."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from collections import Counter\n",
      "authors = Counter()\n",
      "for record in result:\n",
      "    if \"AU\" in record:\n",
      "        for author in record[\"AU\"]:\n",
      "            authors[author] += 1\n",
      "authors.most_common(10) # return the 10 most prolific authors"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 69,
       "text": [
        "[('Liu ZJ', 17),\n",
        " ('Tsai WC', 16),\n",
        " ('Jacquemyn H', 15),\n",
        " ('Hsiao YY', 13),\n",
        " ('Chen HH', 12),\n",
        " ('Nemesio A', 11),\n",
        " ('Dixon KW', 10),\n",
        " ('Chen LJ', 10),\n",
        " ('Pan ZJ', 10),\n",
        " ('Peakall R', 10)]"
       ]
      }
     ],
     "prompt_number": 69
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We could then loop through the entries searching for a particular author with code like the following:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "search_author = \"Liu ZJ\"\n",
      "for record in result:\n",
      "    if not \"AU\" in record:\n",
      "         continue\n",
      "    if search_author in record[\"AU\"]:\n",
      "        print(\"Author %s found: %s\" % (search_author, record[\"SO\"]))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Author Liu ZJ found: Int Rev Cell Mol Biol. 2014;311:157-82. doi: 10.1016/B978-0-12-800179-0.00003-9.\n",
        "Author Liu ZJ found: Mol Phylogenet Evol. 2014 Aug;77:216-22. doi: 10.1016/j.ympev.2014.04.005. Epub 2014 Apr 18.\n",
        "Author Liu ZJ found: Int J Biol Macromol. 2014 Mar;64:420-7. doi: 10.1016/j.ijbiomac.2013.12.024. Epub 2013 Dec 24.\n",
        "Author Liu ZJ found: PLoS One. 2013 May 29;8(5):e64621. doi: 10.1371/journal.pone.0064621. Print 2013.\n",
        "Author Liu ZJ found: PLoS One. 2013 Apr 4;8(4):e60371. doi: 10.1371/journal.pone.0060371. Print 2013.\n",
        "Author Liu ZJ found: PLoS One. 2013;8(4):e60097. doi: 10.1371/journal.pone.0060097. Epub 2013 Apr 5.\n",
        "Author Liu ZJ found: Plant Cell Physiol. 2013 Feb;54(2):e7. doi: 10.1093/pcp/pcs187. Epub 2013 Jan 10.\n",
        "Author Liu ZJ found: PLoS One. 2013;8(1):e53695. doi: 10.1371/journal.pone.0053695. Epub 2013 Jan 7.\n",
        "Author Liu ZJ found: PLoS One. 2012;7(6):e38880. doi: 10.1371/journal.pone.0038880. Epub 2012 Jun 27.\n",
        "Author Liu ZJ found: PLoS One. 2012;7(6):e38788. doi: 10.1371/journal.pone.0038788. Epub 2012 Jun 7.\n",
        "Author Liu ZJ found: PLoS One. 2012;7(5):e37478. doi: 10.1371/journal.pone.0037478. Epub 2012 May 23.\n",
        "Author Liu ZJ found: Plant Cell Physiol. 2012 Jun;53(6):1053-67. doi: 10.1093/pcp/pcs048. Epub 2012 Apr 11.\n",
        "Author Liu ZJ found: PLoS One. 2012;7(1):e29718. doi: 10.1371/journal.pone.0029718. Epub 2012 Jan 10.\n",
        "Author Liu ZJ found: PLoS One. 2011;6(10):e24864. doi: 10.1371/journal.pone.0024864. Epub 2011 Oct 10.\n",
        "Author Liu ZJ found: Mol Phylogenet Evol. 2011 Nov;61(2):308-20. doi: 10.1016/j.ympev.2011.06.006. Epub 2011 Jun 22.\n",
        "Author Liu ZJ found: Plant Cell Physiol. 2011 Mar;52(3):563-77. doi: 10.1093/pcp/pcr015. Epub 2011 Jan 28.\n",
        "Author Liu ZJ found: Plant Cell Physiol. 2011 Feb;52(2):238-43. doi: 10.1093/pcp/pcq201. Epub 2011 Jan 17.\n"
       ]
      }
     ],
     "prompt_number": 70
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Hopefully this section gave you an idea of the power and flexibility of the `Entrez` and `Medline` interfaces and how they can be used together."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"subsec%3Aentrez_example_genbank\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.14.2  Searching, downloading, and parsing Entrez Nucleotide records"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here we\u2019ll show a simple example of performing a remote Entrez query. In section [2.3](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3Aorchids) of the parsing examples, we talked about using NCBI\u2019s Entrez website to search the NCBI nucleotide databases for info on _Cypripedioideae_, our friends the lady slipper orchids. Now, we\u2019ll look at how to automate that process using a Python script. In this example, we\u2019ll just show how to connect, get the results, and parse them, with the Entrez module doing all of the work.\n",
      "\n",
      "First, we use EGQuery to find out the number of results we will get before actually downloading them. EGQuery will tell us how many search results were found in each of the databases, but for this example we are only interested in nucleotides:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.egquery(term=\"Cypripedioideae\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "for row in result[\"eGQueryResult\"]:\n",
      "    if row[\"DbName\"] == \"nuccore\":\n",
      "        print(row[\"Count\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "2677\n"
       ]
      }
     ],
     "prompt_number": 71
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, we expect to find 2677 Entrez Nucleotide records (814 were found in 2008; it is likely to increase further in the future). If you find some ridiculously high number of hits, you may want to reconsider if you really want to download all of them, which is our next step:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "handle = Entrez.esearch(db=\"nucleotide\", term=\"Cypripedioideae\", retmax=500)\n",
      "result = Entrez.read(handle)\n",
      "handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 72
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here, `result` is a Python dictionary containing the search results and some auxiliary information. Just for information, let\u2019s look at what is stored in this dictionary:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result.keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 73,
       "text": [
        "[u'Count',\n",
        " u'RetMax',\n",
        " u'IdList',\n",
        " u'TranslationStack',\n",
        " u'TranslationSet',\n",
        " u'RetStart',\n",
        " u'QueryTranslation']"
       ]
      }
     ],
     "prompt_number": 73
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"Count\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 74,
       "text": [
        "'2677'"
       ]
      }
     ],
     "prompt_number": 74
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "which is the number we expected. The list of ids is stored in record['IdList']; let\u2019s look at the first 5 results:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result[\"IdList\"][:5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 75,
       "text": [
        "['653816630', '653816628', '653816509', '653816507', '653816379']"
       ]
      }
     ],
     "prompt_number": 75
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can download these records using EFetch. While you could download these records one by one, to reduce the load on NCBI\u2019s servers, it is better to fetch a bunch of records at the same time, as shown below. However, in this situation you should ideally be using the history feature described later in Section [9.15](#sec%3Aentrez-webenv). To limit the load on the NCBI servers, we'll only retrieve records for the first 5 UIDs."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "idlist = \",\".join(result[\"IdList\"][:5])\n",
      "handle2 = Entrez.efetch(db=\"nucleotide\", id=idlist, retmode=\"xml\")\n",
      "result2 = Entrez.read(handle2)\n",
      "handle2.close()\n",
      "len(result2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 76,
       "text": [
        "5"
       ]
      }
     ],
     "prompt_number": 76
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Each of these records corresponds to one GenBank record."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result2[0].keys()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 77,
       "text": [
        "[u'GBSeq_moltype',\n",
        " u'GBSeq_source',\n",
        " u'GBSeq_sequence',\n",
        " u'GBSeq_primary-accession',\n",
        " u'GBSeq_definition',\n",
        " u'GBSeq_accession-version',\n",
        " u'GBSeq_topology',\n",
        " u'GBSeq_length',\n",
        " u'GBSeq_feature-table',\n",
        " u'GBSeq_create-date',\n",
        " u'GBSeq_other-seqids',\n",
        " u'GBSeq_division',\n",
        " u'GBSeq_taxonomy',\n",
        " u'GBSeq_comment',\n",
        " u'GBSeq_references',\n",
        " u'GBSeq_update-date',\n",
        " u'GBSeq_organism',\n",
        " u'GBSeq_locus',\n",
        " u'GBSeq_strandedness']"
       ]
      }
     ],
     "prompt_number": 77
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(result2[0][\"GBSeq_primary-accession\"])\n",
      "print(result2[0][\"GBSeq_other-seqids\"])\n",
      "print(result2[0][\"GBSeq_definition\"])\n",
      "print(result2[0][\"GBSeq_organism\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "KF361699\n",
        "['gb|KF361699.1|', 'gi|653816630']\n",
        "Paphiopedilum malipoense hypothetical protein Ycf1 (ycf1) gene, partial cds; chloroplast\n",
        "Paphiopedilum malipoense\n"
       ]
      }
     ],
     "prompt_number": 78
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "You could use this to quickly set up searches \u2013 but for heavy usage, see Section [9.15](#sec%3Aentrez-webenv)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aentrez-search-fetch-genbank\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.14.3  Searching, downloading, and parsing GenBank records"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The GenBank record format is a very popular method of holding information about sequences, sequence features, and other associated sequence information. The format is a good way to get information from the NCBI databases at http://www.ncbi.nlm.nih.gov/.\n",
      "\n",
      "In this example we\u2019ll show how to query the NCBI databases, to retrieve the records from the query, and then parse them using `Bio.SeqIO` - something touched on in Section [5.3.1](http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec%3ASeqIO_GenBank_Online). For simplicity, this example does not take advantage of the WebEnv history feature \u2013 see Section [9.15](#sec%3Aentrez-webenv) for this.\n",
      "\n",
      "First, we want to make a query and find out the ids of the records to retrieve. Here we\u2019ll do a quick search for one of our favorite organisms, _Opuntia_ (prickly-pear cacti). We can do quick search and get back the GIs (GenBank identifiers) for all of the corresponding records.\n",
      "\n",
      "We start by using EGQuery to check how many records there are:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.egquery(term=\"Opuntia AND rpl16\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "for row in result[\"eGQueryResult\"]:\n",
      "    if row[\"DbName\"] == \"nuccore\":\n",
      "        print(row[\"Count\"])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "26\n"
       ]
      }
     ],
     "prompt_number": 79
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next we use ESearch to retrieve the first 20 GenBank identifiers (we could retrieve more by using `retmax`)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "handle2 = Entrez.esearch(db=\"nuccore\", term=\"Opuntia AND rpl16\")\n",
      "result2 = Entrez.read(handle2)\n",
      "handle2.close()\n",
      "gi_list = result2[\"IdList\"]\n",
      "len(gi_list), gi_list"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 80,
       "text": [
        "(20,\n",
        " ['377581039', '330887241', '330887240', '330887239', '330887238', '330887237', '330887236', '330887235', '330887233', '330887232', '330887231', '330887228', '330887227', '330887226', '330887225', '330887224', '330887223', '57240072', '57240071', '6273287'])"
       ]
      }
     ],
     "prompt_number": 80
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now we use EFetch to retrieve the GenBank records associated with these GIs - note that with older versions of Biopython you had to supply a comma separated list of GI numbers to Entrez, as of Biopython 1.59 you can pass a list and this is converted for you:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#gi_str = \",\".join(gi_list)\n",
      "#handle = Entrez.efetch(db=\"nuccore\", id=gi_str, rettype=\"gb\", retmode=\"text\")\n",
      "handle3 = Entrez.efetch(db=\"nuccore\", id=gi_list, rettype=\"gb\", retmode=\"text\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 81
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If you want to look at the raw GenBank files, you can read from this handle and print out the result:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result3 = handle3.read()\n",
      "handle3.close()\n",
      "print(result3)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "LOCUS       HQ621368                 399 bp    DNA     linear   PLN 26-FEB-2012\n",
        "DEFINITION  Opuntia decumbens voucher Martinez & Eggli 146a (ZSS) ribosomal\n",
        "            protein L16 (rpl16) gene, partial cds; chloroplast.\n",
        "ACCESSION   HQ621368\n",
        "VERSION     HQ621368.1  GI:377581039\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia decumbens\n",
        "  ORGANISM  Opuntia decumbens\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 399)\n",
        "  AUTHORS   Arakaki,M., Christin,P.A., Nyffeler,R., Lendel,A., Eggli,U.,\n",
        "            Ogburn,R.M., Spriggs,E., Moore,M.J. and Edwards,E.J.\n",
        "  TITLE     Contemporaneous and recent radiations of the world's major\n",
        "            succulent plant lineages\n",
        "  JOURNAL   Proc. Natl. Acad. Sci. U.S.A. 108 (20), 8379-8384 (2011)\n",
        "   PUBMED   21536881\n",
        "REFERENCE   2  (bases 1 to 399)\n",
        "  AUTHORS   Arakaki,M., Christin,P.-A., Nyffeler,R., Eggli,U., Ogburn,R.M.,\n",
        "            Spriggs,E., Moore,M.J. and Edwards,E.J.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (15-NOV-2010) Department of Ecology and Evolutionary\n",
        "            Biology, Brown University, 80 Waterman St., Providence, RI 02912,\n",
        "            USA\n",
        "COMMENT     ##Assembly-Data-START##\n",
        "            Assembly Method       :: MIRA V3rc4; Geneious v. 4.8\n",
        "            Sequencing Technology :: 454\n",
        "            ##Assembly-Data-END##\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..399\n",
        "                     /organism=\"Opuntia decumbens\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /specimen_voucher=\"Martinez & Eggli 146a (ZSS)\"\n",
        "                     /db_xref=\"taxon:867482\"\n",
        "                     /tissue_type=\"stem\"\n",
        "                     /note=\"authority: Opuntia decumbens Salm-Dyck\"\n",
        "     gene            <1..>399\n",
        "                     /gene=\"rpl16\"\n",
        "     CDS             <1..>399\n",
        "                     /gene=\"rpl16\"\n",
        "                     /codon_start=1\n",
        "                     /transl_table=11\n",
        "                     /product=\"ribosomal protein L16\"\n",
        "                     /protein_id=\"AFB70658.1\"\n",
        "                     /db_xref=\"GI:377581040\"\n",
        "                     /translation=\"NPKRTRFCKQHRGRMKGISYRGNRICFGRYALQALEPAWITSRQ\n",
        "                     IEAGRRAMTRNARRGGKIWVRIFPDKPVTVKSAESRMGSGKGSHLYWVVVVKPGRILY\n",
        "                     EISGVSENIARRAISIAASKMPVRTQFIISG\"\n",
        "ORIGIN      \n",
        "        1 aaccccaaaa gaaccagatt ctgtaaacaa catagaggaa gaatgaaggg aatatcttat\n",
        "       61 cgggggaatc gtatttgttt cggaagatat gctcttcagg cacttgagcc tgcttggatc\n",
        "      121 acgtctagac aaatagaagc aggtcggcga gcaatgacgc gaaatgcacg ccgcggtgga\n",
        "      181 aaaatatggg tacgtatatt tccagacaaa ccagttacag taaaatctgc ggaaagccgt\n",
        "      241 atgggttcgg ggaaaggatc ccacctatat tgggtagttg ttgtcaaacc cggtcgaata\n",
        "      301 ctttatgaaa taagcggagt atcagaaaat atagcccgaa gggctatctc gatagcggca\n",
        "      361 tctaaaatgc ctgtacgaac tcaattcatt atttcagga\n",
        "//\n",
        "\n",
        "LOCUS       HM041482                1197 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Cylindropuntia tunicata ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041482\n",
        "VERSION     HM041482.1  GI:330887241\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Cylindropuntia tunicata\n",
        "  ORGANISM  Cylindropuntia tunicata\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Cylindropuntia.\n",
        "REFERENCE   1  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1197\n",
        "                     /organism=\"Cylindropuntia tunicata\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:766221\"\n",
        "     gene            <1..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1094..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gtgatatacg aaacagtaag agcccatagt atgaagtatg aactaataac tatagaacta\n",
        "       61 ataaccaact catcgcatca cattatctgg atccaaagaa gcagtcaaga taggatattt\n",
        "      121 tggtcctatc attgcagcaa ctgaattttt tttttcataa acaagaaatc gaatgagttg\n",
        "      181 tcaagcaaaa gaaaaaaaaa aaaagaaaaa tatacnttaa aggaggggga tgcggataaa\n",
        "      241 tggaaaggcg aaagaaagaa aaaaatgaat ctaaatgata tacgattcca ctatgtaagg\n",
        "      301 tctttgaatc atatcataaa agacaatgta ataaagcatg aatacagatt cacacataat\n",
        "      361 tatctgatat gaatctattc atagaaaaaa gaaaaaagta agagcctccg gccaataaag\n",
        "      421 actaagaggg gttggctcaa aaacaaagtt cattaagagc tcccattgta gaattcagac\n",
        "      481 ctaatcatta atcaagaagc gatgggaacg atgtaatcca tgaatacaga agattcaatt\n",
        "      541 gaaaaaagaa tcctaatgat tcattgggga ggatggcgga acgaaccaga gaccaattca\n",
        "      601 tctattctga aaagtgataa actaatccta taaaactaaa atagatattg aaagagtaaa\n",
        "      661 tattcgcccg cgaaaattcc ttttttatta aattgctcat attttatttt agcaatgcaa\n",
        "      721 tctaataaaa tatatctata caaaaaaaca tagacaaact atatatataa tatttcaaat\n",
        "      781 tcccttatat atccaaatat aaaaatatct aataaattag atgaatatca aagaatctat\n",
        "      841 tgatttagtg tattattaaa tgtatatctt aattcaatat tattattcta ttcattttta\n",
        "      901 ttcattttca aatttataat atattaatct atatattaat ttagaattct attctaattc\n",
        "      961 gaattcaatt tttaaatatt cattcatatt caattaaaat tgaaattttt tcattcgcga\n",
        "     1021 ggagccggat gagaagaaac tctcatgtcc ggttctgtag tagagatgga attaagaaaa\n",
        "     1081 aaccatcaac tataacccca aaagaaccag attctgtaaa caacatagag gaagaatgaa\n",
        "     1141 gggaatatct tatcggggga atcgtatttg tttcggaaaa tatgctctca ggcacga\n",
        "//\n",
        "\n",
        "LOCUS       HM041481                1200 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia palmadora ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041481\n",
        "VERSION     HM041481.1  GI:330887240\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia palmadora\n",
        "  ORGANISM  Opuntia palmadora\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1200)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1200)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1200\n",
        "                     /organism=\"Opuntia palmadora\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001118\"\n",
        "     gene            <1..>1200\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1098..>1200\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 tgatatacga aaagtaagag cccatagtat gaagtatgaa ctaataacta tagaactaat\n",
        "       61 aaccaactca tcgcatcaca ttatctggat ccaaagaagc agtcaagata ggatattttg\n",
        "      121 gtcctatcat tgcagcaact gaattttttt ttcataaaca agaaatcaaa tgagttgtca\n",
        "      181 agcaaaagaa aaaaaaaaga aaaatatacn ttaaaggagg gggatgcgga taaatggaaa\n",
        "      241 ggcgaaagaa agaaaaaaat gaatctaaat gatatacgat tccactatgt aaggtctttg\n",
        "      301 aatcatatca taaaagacaa tgtaataaag catgaataca gattcacaca taattatctg\n",
        "      361 atatgaatct attcatagaa aaaagaaaaa agtaagagcc tccgggccaa taaagactaa\n",
        "      421 gagggttggg ctcaagaaca aagttcatta agagctccat tgtagaattc agacctaatc\n",
        "      481 attaatcaag aagcgatggg aacgatgtaa tccatgaata cagaagattc aattgaaaaa\n",
        "      541 gaatcctaat gattcattgg gaaggatggc ggaacgaacc agagaccaat tcatctattc\n",
        "      601 tgaaaagtga taaactaatc ctataaaact aaaatagata ttgaaagagt aaatattcgc\n",
        "      661 ccgcgaaaat tcctttttta ttaaattgct cacattttat tttagcaatg caatctaata\n",
        "      721 aaatatatct atacaaaaaa atatagacaa actatatata taatatattt caaatttcct\n",
        "      781 tatatatcct aatataaaaa tatctaataa attagatgaa tatcaaagaa tctattgatt\n",
        "      841 tagtgtatta ttaaatgtat atcttaattc aatattatta ttctattcat ttttattatt\n",
        "      901 catttttatt cattttcaaa tttagaatat attaatctat atattaattt agaattctat\n",
        "      961 tctaattcga attcaatttt taaatattca tattcaatta aaattgaaat tttttcattc\n",
        "     1021 gcgaggagcc ggatgagaag aaactctcac gtccggttct gtagtagagg tggaattaag\n",
        "     1081 aaaaaaccat caactataac cccaaaagaa ccagattctg taaacaacat agaggaagaa\n",
        "     1141 tgaagggaat atcttatcgg gggaatcgta tttgtttcgg aagatatgct ctcagcacga\n",
        "//\n",
        "\n",
        "LOCUS       HM041480                1153 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia microdasys ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041480\n",
        "VERSION     HM041480.1  GI:330887239\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia microdasys\n",
        "  ORGANISM  Opuntia microdasys\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1153)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1153)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1153\n",
        "                     /organism=\"Opuntia microdasys\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:169217\"\n",
        "     gene            <1..>1153\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1079..>1153\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gcccatagta tgaagtatga actaataact atagaactaa taaccaactc atcgcatcac\n",
        "       61 attatctgga tccaaagaag cagtcaagat aggatatttt ggtcctatca ttgcagcaac\n",
        "      121 tgaatttttt ttttcataaa caagaaatca aatgagttgt caagcaaaag aaaaaaaaaa\n",
        "      181 aaaaaaatat actttaaggg ggggggatgg ggataaaggg aaaggggaaa aaaaaaaaaa\n",
        "      241 aatgaatcta aatgatatac aattccacta tgaaaggtct ttgaatcata tcaaaaaaaa\n",
        "      301 caatgtaata aagcaggaat acagattccc acataattat ctgatatgaa tcttttcata\n",
        "      361 aaaaaaaaaa aaaagtaaga gcctccggcc aataaagact aagagggttg gctcaagaac\n",
        "      421 aaagttcatt aagggctcca ttgtagaatt cagacctaat cattaatcaa gaggcgatgg\n",
        "      481 gaacgatgta atccatgaat acagaagatt caattgaaaa agaatcctaa tgattcattg\n",
        "      541 ggaaggatgg cggaacgaac cagagaccaa ttcatctatt ctgaaaagtg aaaaactaat\n",
        "      601 cctataaaac taaaatagat attgaaagag taaatattcg cccgcgaaaa ttcctttttt\n",
        "      661 attaaattgc tcacatttta ttttagcaat gcaatctaat aaaatatatc tatacaaaaa\n",
        "      721 aatatagaca aactatatat ataatatatt tcaaatttcc ttatatatcc taatataaaa\n",
        "      781 atatctaata aattagatga atatcaaaga atctattgat ttagtgtatt attaaatgta\n",
        "      841 tatcttaatt caatattatt attctattca tttttattat tcatttttat tcattttcaa\n",
        "      901 atttagaata tattaatcta tatattaatt tataattcta ttctaattcg aattcaattt\n",
        "      961 ttaaatattc atattcaatt aaaattgaaa ttttttcatt cgcgaggagc cggatgagaa\n",
        "     1021 gaaactctca cgtccggttc tgtagtagag gtggaattaa gaaaaaacca tcaactataa\n",
        "     1081 ccccaaaaga accagattct gtaaacaaca tagaggaaga atgaagggaa tatcttatcg\n",
        "     1141 ggggatatcg tat\n",
        "//\n",
        "\n",
        "LOCUS       HM041479                1197 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia megasperma ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041479\n",
        "VERSION     HM041479.1  GI:330887238\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia megasperma\n",
        "  ORGANISM  Opuntia megasperma\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1197\n",
        "                     /organism=\"Opuntia megasperma\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001117\"\n",
        "     gene            <1..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1098..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gatatacgaa aagtaagagc ccatagtatg aagtatgaac taataactat agaactaata\n",
        "       61 accaactcat cgcatcacat tatccggatc caaagaagca gtcaagatag gatattttgg\n",
        "      121 tcctatcatt gcagcaactg aatttttttt tcataaacaa gaaatcaaat gagttgtcaa\n",
        "      181 gcaaaagaaa aaaaaaaaag aaaaatatac tttaaaggag ggggatgcgg ataaatggaa\n",
        "      241 aggcgaaaga aagaaaaaaa tgaatctaaa tgatatacga ttccnctatg taaggtcttt\n",
        "      301 gaatcatatc ataaaagaca atgtaataaa gcatgaatac agattcacac ataattatct\n",
        "      361 gatatgaatc tattcataga aaaaagaaaa aagtaagagc ctccgggcca ataaagacta\n",
        "      421 agagggttgg ctcaagaaca aagttcatta agagctccat tgtagaattc agacctaatc\n",
        "      481 attaatcaag aagcgatggg aacgatgtaa tccatgaata cagaagattc aattgaaaaa\n",
        "      541 gaatcctaat gattcattgg gaaggatggc ggaacgaacc agagaccaat tcatctattc\n",
        "      601 tgaaaagtga taaactaatc ctataaaact aaaatagata ttgaaagagt aaatattcgc\n",
        "      661 ccgcgaaaat tcctttttta ttaaattgct cacattttat tttagcaatg caatctaata\n",
        "      721 aaatatatct atacaaaaaa atatagacaa actatatata taatatattt caaatttcct\n",
        "      781 tatatatcct aatataaaaa tatctaataa attagatgaa tatcaaagaa tctattgatt\n",
        "      841 tagtgtatta ttaaatgtat atcttaattc aatattttta ttctattcat ttttattatt\n",
        "      901 catttttatt cattttcaaa tttagaatat attaatctat atattaattt agaattctat\n",
        "      961 tctaattcga attcaatttt taaatattca tattcaatta aaattgaaat tttttcattc\n",
        "     1021 gcgaggagcc ggatgagaag aaactctcac gtccggttct gtagtagagg tggaattaag\n",
        "     1081 aaaaaaccat caactataac cccaaaagaa ccagattctg taaacaacat agaggaagaa\n",
        "     1141 tgaagggaat atcttatcgg gggaatcgta tttgtttcgg aagatatgct ctcagca\n",
        "//\n",
        "\n",
        "LOCUS       HM041478                1187 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia macbridei ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041478\n",
        "VERSION     HM041478.1  GI:330887237\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia macbridei\n",
        "  ORGANISM  Opuntia macbridei\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1187)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1187)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1187\n",
        "                     /organism=\"Opuntia macbridei\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001116\"\n",
        "     gene            <1..>1187\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1090..>1187\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 aaaagtaaga gcccatagta tgaagtatga actaataact atagaactaa taaccaactc\n",
        "       61 atcgcatcac attatctgga tccaaagaag cagtcaagat aggatatttt ggtcctatca\n",
        "      121 ttgcagcaac tgaatttttt tttcataaac aagaaatcaa atgagttgtc aagcaaaaga\n",
        "      181 aaaaaaaaaa agaaaaatat acattaaagg agggggatgc ggataaatgg aaaggcgaaa\n",
        "      241 gaaagaaaaa aatgaatcta aatgatatac gattccacta tgtaaggtct ttgaatcata\n",
        "      301 tcataaaaga caatgtaata aagcatgaat acagattcac acataattat ctgatatgaa\n",
        "      361 tctattcata gaaaaaagaa aaaagtaaga gcctccggcc aataaagact aagagggttg\n",
        "      421 gctcaagaac aaagttcatt aagggctcca tttgtagaat tcagacctaa tcattaatca\n",
        "      481 agaagcgatg ggaacgatgt aattccatga atacagaaga ttcaattgaa aaagatccta\n",
        "      541 atgattcatt gggaaggatg gcggacgaac cagagaccaa ttcatctatt ctgaaaagtg\n",
        "      601 ataaactaat cctataaaac taaaatagat attgaaagag taaatattcg cccgcgaaaa\n",
        "      661 ttcctttttt attaaattgc tcacatttta ttttagcaat gcaatctaat aaaatatatc\n",
        "      721 tatacaaaaa aaatatagac aaactatata tataatatat ttcaaatttc cttatatatc\n",
        "      781 ctaatataaa aatatctaat aatttagatg aatatcaaag aatctattga tttagtgtat\n",
        "      841 tattaaatgt atatcttaat tcaatattat tattctattc atttttatta ttcattttta\n",
        "      901 ttcattttca aatttagaat atattaatct atatattaat ttagaattct attctaattc\n",
        "      961 gaattcaatt tttaaatatt catattcaat taaaattgaa attttttcat tcgcgaggag\n",
        "     1021 ccggatgaga agaaactctc acgtccggtt ctgtagtaga ggtggaatta agaaaaaacc\n",
        "     1081 atcaactata accccaaaag aaccagattc tgtaaacaac atagaggaag aatgaaggga\n",
        "     1141 atatcttatc gggggaatcg tatttgtttc ggaagatatg ctctcag\n",
        "//\n",
        "\n",
        "LOCUS       HM041477                1197 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Cylindropuntia leptocaulis ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041477\n",
        "VERSION     HM041477.1  GI:330887236\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Cylindropuntia leptocaulis\n",
        "  ORGANISM  Cylindropuntia leptocaulis\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Cylindropuntia.\n",
        "REFERENCE   1  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1197)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1197\n",
        "                     /organism=\"Cylindropuntia leptocaulis\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:866983\"\n",
        "     gene            <1..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1096..>1197\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 ttgtgngnct cctgaagagt aggagcccct agtatgaagt atgaactaat aactatagaa\n",
        "       61 ctaataacca actcatcgca tcacattatc cggatccaaa aaagcagtca agataggata\n",
        "      121 ttttggtcct atcattgcag caactgaatt ttttttttca taaacaagaa atcgaatgag\n",
        "      181 ttgtcaagca aaagaaaaaa aaagaaaaat atactttaaa ggagggggat gcggataaat\n",
        "      241 ggaaaggcga aagaaagaaa aaaatgaatc taaatgatat aggattcccc tatgtaaggt\n",
        "      301 ctttgaatca tatcataaaa gacaatgtaa taaagcatga atacagattc ccacataatt\n",
        "      361 atctgatatg aatctattcc tagaaaaaag aaaaaagtaa gagcctccgg ccaataaaga\n",
        "      421 ctaagagggt tggctcaaga acaaagttca ttaaaagctc ccttgtagaa ttcagaccta\n",
        "      481 atcnttaatc aagaagcgat gggaacgatg taatccctga atacagaaga ttcaattgaa\n",
        "      541 aaagaatcct aatgattcat tgggaaggat ggcggaacga accagagacc aattcatcta\n",
        "      601 ttctgaaaag tgataaacta atcctataaa actaaaatag atattgaaag agtaaatatt\n",
        "      661 cgcccgcgaa atttcctttt ttattaaatt gctcatattt ttttttagca atgcaatcta\n",
        "      721 ataaaatata tctctacaaa aaaacataga caaactatat atatatatat atataatatt\n",
        "      781 tcaaattccc ttatatatcc aaatataaaa atatctaata aattagatga atatcaaaga\n",
        "      841 atctattgat ttagtgtatt attaaatgta tatcttaatt caatattatt attctattca\n",
        "      901 tttttattca ttttcaaatt tataatatat taatctatat attaatatag aattctattc\n",
        "      961 taattcgaat tcaattttta aatattcata ttcaattaaa attgaaattt tttcattcgc\n",
        "     1021 gaggagccgg atgagaagaa actctcatgt ccggttctgt agtagagatg gaattaagaa\n",
        "     1081 aaaaccatca actataaccc caaaagaacc ggattctgta aacaacatag aggaagaatg\n",
        "     1141 aagggaatat cttgtcgggg gaatcgatnn gtncggaant natgntcgcn gcgcgcc\n",
        "//\n",
        "\n",
        "LOCUS       HM041476                1205 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia lasiacantha ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041476\n",
        "VERSION     HM041476.1  GI:330887235\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia lasiacantha\n",
        "  ORGANISM  Opuntia lasiacantha\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1205)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1205)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1205\n",
        "                     /organism=\"Opuntia lasiacantha\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:547104\"\n",
        "     gene            <1..>1205\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1103..>1205\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gggcccnnna ngangaaaag tagagcccat agtatgaagt atgaactaat aactatagaa\n",
        "       61 ctaataacca actcatcgca tcacattatc tggatccaaa gaagcagtca agataggata\n",
        "      121 ttttggtcct atcattgcag caactgaatt ttttttttca taaacaagaa atcaaatgag\n",
        "      181 ttgtcaagca aaagaaaaaa aaaaagaaaa atatccttta aaggaggggg atgcggataa\n",
        "      241 atggaaaggc gaaagaaaga aaaaaatgaa tctaaatgat atacgattcc cctatgtaag\n",
        "      301 gtctttgaat catatcataa aagacaatgt aataaagcat gaatacagat tcccccataa\n",
        "      361 ttatctgata tgaatctatt cctagaaaaa agaaaaaagt aagagcctcc ggccaataaa\n",
        "      421 gactaagagg gttggctcaa gaacaaagtt cattaagggc tccattgtag aattcagacc\n",
        "      481 taatcattaa tcaagaggcg atgggaacga tgtaatccat gaatacagaa gattcaattg\n",
        "      541 aaaaagaatc ctaatgattc attgggaagg atggcggaac gaaccagaga ccaattcatc\n",
        "      601 tattctgaaa agtgataaac taatcctata aaactaaaat agatattgaa agagtaaata\n",
        "      661 ttcgcccgcg aaaattcctt ttttattaaa ttgctcacat tttattttag caatgcaatc\n",
        "      721 taataaaatc tatctataca aaaaaatata gacaaactat atatataata tatttcaaat\n",
        "      781 ttccttatat atcctaatat aaaaatatct aataaattag atgaatatca aagaatctat\n",
        "      841 tgatttagtg tattattaaa tgtatatctt aattcaatat tattattcta ttcattttta\n",
        "      901 ttattcattt ttattcattt tcaaatttag aatatattaa tctatatatt aatttataat\n",
        "      961 tctattctaa ttcgaattca atttttaaat attcatattc aattaaaatt gaaatttttt\n",
        "     1021 cattcgcgag gagccggatg agaagaaact ctcacgtccg gttctgtagt agaggtggaa\n",
        "     1081 ttaagaaaaa accatcaact ataaccccaa aagaaccaga ttctgtaaac aacatagagg\n",
        "     1141 aagaatgaag ggaatatctt atcgagggaa tcgtatttgt ttcggaagat agtnctngcn\n",
        "     1201 nggtg\n",
        "//\n",
        "\n",
        "LOCUS       HM041474                1163 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia helleri ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041474\n",
        "VERSION     HM041474.1  GI:330887233\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia helleri\n",
        "  ORGANISM  Opuntia helleri\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1163)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1163)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1163\n",
        "                     /organism=\"Opuntia helleri\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001115\"\n",
        "     gene            <1..>1163\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1081..>1163\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gagcccatag tatgaagtat gaactaataa ctatagaact aataaccaac tcatcgcatc\n",
        "       61 acattatccg gatccaaaga agcagtcaag ataggatatt ttggtcctat cattgcagca\n",
        "      121 actgaatttt tttttcataa acaagaaatc aaatgagttg tcaagcaaaa gaaaaaaaaa\n",
        "      181 aaagaaaaat atacattaaa ggagggggat gcggataaat ggaaaggcga aagaaagaaa\n",
        "      241 aaaatgaatc taaatgatat acgattccnc tatgtaaggt ctttgaatca tatcataaaa\n",
        "      301 gacaatgtaa taaagcatga atacagattc acacataatt atctgatatg aatctattca\n",
        "      361 tagaaaaaag aaaaaagtaa gagcctccgg ccaataaaga ctaagagggt tggctcaaga\n",
        "      421 acaaagttca ttaagggctc cattgtagaa ttcagaccta atcattaatc aagaagcgat\n",
        "      481 gggaacgatg taatccatga atacagaaga ttcaattgaa aaagaatcct aatgattcat\n",
        "      541 tgggaaggat ggcggaacga accagagacc aattcatcta ttctgaaaag tgataaacta\n",
        "      601 atcctataaa actaaaatag atattgaaag agtaaatatt cgcccgcgaa aattcctttt\n",
        "      661 ttattaaatt gctcacattt tattttagca atgcaatcta ataaaatata tctatacaaa\n",
        "      721 aaaatataga caaactatat atataatata tttaaaattt ccttatatat cctaatataa\n",
        "      781 aaatatctaa taaattagat gaatatcaaa gaatctattg atttagtgta ttattaaatg\n",
        "      841 tatatcttaa ttcaatattt ttattctatt catttttatt attcattttt attcattttc\n",
        "      901 aaatttagaa tatattaatc tatatattaa tttagaattc tattctaatt cgaattcaat\n",
        "      961 ttttaaatat tcatattcaa ttaaaattga aattttttca ttcgcgagga gccggatgag\n",
        "     1021 aagaaactct cacgtccggt tctgtagtag aggtggaatt aagaaaaaac catcaactat\n",
        "     1081 aaccccaaaa gaaccagatt ctgtaaacaa catagaggaa gaatgaaggg aatatcttat\n",
        "     1141 cgggggaatc gtatttgttt cgg\n",
        "//\n",
        "\n",
        "LOCUS       HM041473                1203 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia excelsa ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041473\n",
        "VERSION     HM041473.1  GI:330887232\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia excelsa\n",
        "  ORGANISM  Opuntia excelsa\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1203)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1203)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1203\n",
        "                     /organism=\"Opuntia excelsa\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:867487\"\n",
        "     gene            <1..>1203\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1103..>1203\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 ccgnncnttg nnanacagaa nagtagagcc cnttntntga agtatgaact aatcactatt\n",
        "       61 gaactaatcc ccnactcatc gcatcacatt atctggatcc aaagaagcag tcaagatagg\n",
        "      121 atattttggt cctatcattg cagcaactga attttttttt tcctaaacaa gaaatcaaat\n",
        "      181 gagttgtcaa gcaaaagaaa aaaaagaaaa atatacatta aaggaggggg atgcggataa\n",
        "      241 atggaaaggc gaaagaaaga aaaaaatgaa tctaaatgat atacgattcc cctatgtaag\n",
        "      301 gtctttgaat catatcataa aagacaatgt aataaagcat gaatacagat tcccacataa\n",
        "      361 ttatctgata tgaatctatt catagaaaaa agaaaaaagt aagagcctcc ggccaataaa\n",
        "      421 gactaaaagg gttggctcaa gaacaaagtt cattaagggc tccattgtaa aattcagacc\n",
        "      481 taatcattaa tcaagaggcg atgggaacga tgtaatccat gaatacagaa gattcaattg\n",
        "      541 aaaaagaatc ctaatgattc attgggaagg atggcggaac gaaccagaga ccaattcatc\n",
        "      601 tattctgaaa agtgataaac taatcctata aaactaaaat agatattgaa agagtaaata\n",
        "      661 ttcgcccgcg aaaattcctt ttttattaaa ttgctcacat tttattttag caatgcaatc\n",
        "      721 taataaaatc tatctataca aaaaaatata gacaaactat atatataata tatttcaaat\n",
        "      781 ttccttatat atcctaatat aaaaatatct aataaattag atgaatatca aagaatctat\n",
        "      841 tgatttagtg tattattaaa tgtatatctt aattcaatat tattattcta ttcattttta\n",
        "      901 ttattcattt ttattcattt tcaaatttag aatatattaa tctatatatt aatttagaat\n",
        "      961 tctattctaa ttcgaattca atttttaaat attcatattc aattaaaatt gaaatttttt\n",
        "     1021 cattcgcgag gagccggatg agaagaaact ctcacgtccg gttctgtagt agaggtggaa\n",
        "     1081 ttaagaaaaa accatcaact ataaccccaa aagaaccaga ttctgtaaac aacatagagg\n",
        "     1141 aagaatgaag ggaatatctt atcgggggaa tcgtatngtg cnggctngtg cancgcgggc\n",
        "     1201 nng\n",
        "//\n",
        "\n",
        "LOCUS       HM041472                1182 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Opuntia echios ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041472\n",
        "VERSION     HM041472.1  GI:330887231\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia echios\n",
        "  ORGANISM  Opuntia echios\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 1182)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1182)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1182\n",
        "                     /organism=\"Opuntia echios\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:412453\"\n",
        "     gene            <1..>1182\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1085..>1182\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gtaagagccc atagtatgaa gtatgaacta ataactatag aactaataac caactcatcg\n",
        "       61 catcacatta tccggatcca aagaagcagt caagatagga tattttggtc ctatcattgc\n",
        "      121 agcaactgaa tttttttttc ataaacaaga aatcaaatga gttgtcaagc aaaagaaaaa\n",
        "      181 aaaaaaaaaa aaatatacat taaaggaggg ggatgcggat aaatggaaag gcgaaagaaa\n",
        "      241 gaaaaaaatg aatctaaatg atatacgatt ccactatgta aggtctttga atcatatcat\n",
        "      301 aaaagacaat gtaataaagc atgaatacag attcacacat aattatctga tatgaatcta\n",
        "      361 ttcatagaaa aaagaaaaaa gtaagagcct ccggccaata aagactaaga ggttgggctc\n",
        "      421 aagaacaaag ttcattaagg gctccattgt agaattcaga cctaatcatt aatcaagaag\n",
        "      481 cgatgggaac gatgtaatcc atgaatacag aagattcaat tgaaaaagaa tcctaatgat\n",
        "      541 tcattgggaa ggatggcgga acgaaccaga gaccaattca tctattctga aaagtgataa\n",
        "      601 actaatccta taaaactaaa atagatattg aaagagtaaa tattcgcccg cgaaaattcc\n",
        "      661 ttttttatta aattgctcac attttatttt agcaatgcaa tctaataaaa tatatctata\n",
        "      721 caaaaaaata tagacaaact atatatataa tatatttcaa atttccttat atatcctaat\n",
        "      781 ataaaaatat ctaataaatt agatgaatat caaagaatct attgatttag tgtattatta\n",
        "      841 aatgtatatc ttaattcaat attattattc tattcatttt tattattcat ttttattcat\n",
        "      901 tttcaaattt agaatatatt aatctatata ttaatttaga attctattct aattcgaatt\n",
        "      961 caatttttaa atattcatat tcaattaaaa ttgaaatttt ttcattcgcg aggagccgga\n",
        "     1021 tgagaagaaa ctctcacgtc cggttctgta gtagaggtgg aattaagaaa aaaccatcaa\n",
        "     1081 ctataacccc aaaagaacca gattctgtaa acaacataga ggaagaatga agggaatatc\n",
        "     1141 ttatcggggg aatcgtattt gtttcggaag atggctacta ta\n",
        "//\n",
        "\n",
        "LOCUS       HM041469                1189 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea sp. THH-2011 ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041469\n",
        "VERSION     HM041469.1  GI:330887228\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia sp. THH-2011\n",
        "  ORGANISM  Opuntia sp. THH-2011\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1189)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1189)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1189\n",
        "                     /organism=\"Opuntia sp. THH-2011\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001114\"\n",
        "     gene            <1..>1189\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1096..>1189\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 atatacgaaa aagtagagcc catagtatga agtatgaact aataactata gaactaataa\n",
        "       61 ccaactcatc gcatcacatt atctggatcc aaagaagcag tcaagatagg atattttggt\n",
        "      121 cctatcattg cagcaactga attttttttt cataaacaag aaatcaaatg agttgtcaag\n",
        "      181 caaaagaaaa aaaaaaaaga aaaatatact ttaagggagg gggatgcgga taaatggaaa\n",
        "      241 ggcgaaagaa agaaaaaaat gaatctaaat gatatacgat tccactatgt aaggtctttg\n",
        "      301 aatcatatca taaaagacaa tgtaataaag catgaataca gattcacaca taattatctg\n",
        "      361 gtatgaatct attcatagaa aaaagaaaaa agtaagaccc tccggccaat aaagactaag\n",
        "      421 agggttggct caagaacaaa gttcattaag ggctccattg tagaattcag acctaatcat\n",
        "      481 taatcaagaa gcgatgggaa cgatgtaatc catgaataca gaagattcaa ttgaaaaaga\n",
        "      541 atcctaatga ttcattggga aggatggcgg aacgaaccag agaccaattc atctattctg\n",
        "      601 aaaagtgata aactaatcct ataaaactaa aatagatatt gaaagagtaa atattcgccc\n",
        "      661 gcgaaaattc cttttttatt aaattgctca cattttattt tagcaatgca atctaataaa\n",
        "      721 atatatctat acaaaaaaat atagacaaac tatatatata atatatttca aatttcctta\n",
        "      781 tatatcctaa tataaaaata tctaataaat tagatgaata tcaaagaatc tattgattta\n",
        "      841 gtgtattatt aaatgtatat cttaattcaa tattattatt ctattcattt ttattattca\n",
        "      901 tttttattca ttttcaaatt tagaatatat taatctatat attaatttag aattctattc\n",
        "      961 taattcgaat tcaattttta aatattcata ttcaattaaa attgaaattt tttcattcgc\n",
        "     1021 gaggagccgg atgagaagaa actctcacgt ccggttctgt agtagaggtg gaattaagaa\n",
        "     1081 aaaaccatca actataaccc caaaagaacc agattctgta aacaacatag aggaagaatg\n",
        "     1141 aagggaatat cttatcgggg gaatcgtatt tgtttcggaa gatatgctc\n",
        "//\n",
        "\n",
        "LOCUS       HM041468                1202 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea lutea ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041468\n",
        "VERSION     HM041468.1  GI:330887227\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia lutea\n",
        "  ORGANISM  Opuntia lutea\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1202)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1202)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1202\n",
        "                     /organism=\"Opuntia lutea\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001113\"\n",
        "     gene            <1..>1202\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1099..>1202\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gatatacgaa aaagtaagag cccatagtat gaagtatgaa ctaataacta tagaactaat\n",
        "       61 aaccaactca tcgcatcaca ttatctggat ccaaagaagc agtcaagata ggatattttg\n",
        "      121 gtcctatcat tgcagcaact gaattttttt ttcataaaca agaaatcaaa tgagttgtca\n",
        "      181 agcaaaagaa aaaaaaaaaa gaaaaatata ctttaaggga gggggatgcg gataaatgga\n",
        "      241 aaggcgaaag aaagaaaaaa atgaatctaa atgatatacg attcccccta tgtaaggtct\n",
        "      301 ttgaatcata tcataaaaga caatgtaata aagcatgaat acagattcac acataattat\n",
        "      361 ctgatatgaa tctattcata gaaaaaagaa aaaagtaaga ccctccggcc aataaagact\n",
        "      421 aagagggttg gctcaagaac aaagttcatt aagggctcca ttgtagaatt cagacctaat\n",
        "      481 cattaatcaa gaagcgatgg gaacgatgta atccatgaat acagaagatt caattgaaaa\n",
        "      541 agaatcctaa tgattcattg ggaaggatgg cggaacgaac cagagaccaa ttcatctatt\n",
        "      601 ctgaaaagtg ataaactaat cctataaaac taaaatagat attgaaagag taaatattcg\n",
        "      661 cccgcgaaaa ttcctttttt attaaattgc tcacatttta ttttagcaat gcaatctaat\n",
        "      721 aaaatatatc tatacaaaaa aatatagaca aactatatat ataatatatt tcaaatttcc\n",
        "      781 ttatatatcc taatataaaa atatctaata aattagatga atatcaaaga atctattgat\n",
        "      841 ttagtgtatt attaaatgta tatcttaatt caatattatt attctattca tttttattat\n",
        "      901 tcatttttat tcattttcaa atttagaata tattaatcta tatattaatt tagaattcta\n",
        "      961 ttctaattcg aattcaattt ttaaatattc atattcaatt aaaattgaaa ttttttcatt\n",
        "     1021 cgcgaggagc cggatgagaa gaaactctca cgtccggttc tgtagtagag gtggaattaa\n",
        "     1081 gaaaaaacca tcaactataa ccccaaaaga accagattct gtaaacaaca tagaggaaga\n",
        "     1141 atgaagggaa tatcttatcg ggggaatcgt atttgtttcg gaagatatgc tctcaggcac\n",
        "     1201 ga\n",
        "//\n",
        "\n",
        "LOCUS       HM041467                1199 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea karwinskiana ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041467\n",
        "VERSION     HM041467.1  GI:330887226\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia karwinskiana\n",
        "  ORGANISM  Opuntia karwinskiana\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1199)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1199)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1199\n",
        "                     /organism=\"Opuntia karwinskiana\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001112\"\n",
        "     gene            <1..>1199\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1098..>1199\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gtgatatcga aaaagtagag cccatagtat gaagtatgaa ctaataacta tagaactaat\n",
        "       61 aaccaactca tcgcatcaca ttatctggat ccaaagaagc agtcaagata ggatattttg\n",
        "      121 gtcctatcat tgcagcaact gaattttttt ttcataaaca agaaatcaaa tgagttgtca\n",
        "      181 agcaaaagaa aaaaaaaaaa gaaaaattta ctttaaggga gggggatgcg gataaatgga\n",
        "      241 aaggcgaaag aaagaaaaaa atgaatctaa atgatatacg attcccctat gtagggtctt\n",
        "      301 tgaatcatat cataaaaaac aatgtaataa agcatgaata cagattcccc cataattatc\n",
        "      361 tggtatgaat cttttcatag aaaaaaaaaa aaagtaagag cctccggcca ataaaaacta\n",
        "      421 aaagggttgg ctcaagaaca aagttcatta agggctccat tgtagaattc agacctaatc\n",
        "      481 nttaatcaag aagcgatggg aacgatgtaa tccatgaata cagaagattc aattgaaaaa\n",
        "      541 gaatcctaat gattcattgg gaaggatggc ggaacgaacc agagaccaat tcatctattc\n",
        "      601 tgaaaagtga taaactaatc ctataaaact aaaatagata ttgaaagagt aaatattcgc\n",
        "      661 ccgcgaaaat tcctttttta ttaaattgct cacattttat tttagcaatg caatctaata\n",
        "      721 aaatatatct atacaaaaaa atatagacaa actatatata taatatattt caaatttcct\n",
        "      781 tatatatcct aatataaaaa tatctaataa attagatgaa tatcaaagaa tctattgatt\n",
        "      841 tagtgtatta ttaaatgtat atcttaattc aatattatta ttctattcat ttttattatt\n",
        "      901 catttttatt cattttcaaa tttagaatat attaatctat atattaattt agaattctat\n",
        "      961 tctaattcga attcaatttt taaatattca tattcaatta aaattgaaat tttttcattc\n",
        "     1021 gcgaggagcc ggatgagaag aaactctcac gtccggttct gtagtagagg tggaattaag\n",
        "     1081 aaaaaaccat caactataac cccaaaagaa ccagattctg taaacaacat agaggaagaa\n",
        "     1141 tgaagggaat atcttatgcg ggggaatcgt attgtttcgg aagatatgct ctgcggccc\n",
        "//\n",
        "\n",
        "LOCUS       HM041466                1205 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea gaumeri ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041466\n",
        "VERSION     HM041466.1  GI:330887225\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Nopalea gaumeri\n",
        "  ORGANISM  Nopalea gaumeri\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1205)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1205)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1205\n",
        "                     /organism=\"Nopalea gaumeri\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001111\"\n",
        "     gene            <1..>1205\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1103..>1205\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gctgtgatat acgaaanagt aagagcccat agtatgaagt atgaactaat aactatagaa\n",
        "       61 ctaataacca actcatcgca tcacattatc tggatccaaa gaagcagtca agataggata\n",
        "      121 ttttggtcct atcattgcag caactgaatt tttttttcat aaacaagaaa tcaaatgagt\n",
        "      181 tgtcaagcaa aagaaaaaaa aaaaaaaaaa tatacattaa aggaggggga tgcggataaa\n",
        "      241 tggaaaggcg aaagaaagaa aaaaatgaat ctaaatgata tacgattcca ctatgtaagg\n",
        "      301 tctttgaatc atatcataaa agacaatgta ataaagcatg aatacagatt cacacataat\n",
        "      361 tatctgaata tgaatctatt catagaaaaa agaaaaaagt aagaccctcc ggccaataaa\n",
        "      421 gactaaaggg gttggctcaa gaacaaagtt cattaagggc tccattgtag aattcagacc\n",
        "      481 taatcattaa tcaagaagcg atgggaacga tgtaatccat gaatacagaa gattcaattg\n",
        "      541 aaaaagaatc ctaatgattc attgggaagg atggcggaac gaaccagaga ccaattcatc\n",
        "      601 tattctgaaa agtgataaac taatcctata aaactaaaat agatattgaa agagtaaata\n",
        "      661 ttcgcccgcg aaaattcctt ttttattaaa ttgctcacat tttattttag caatgcaatc\n",
        "      721 taataaaata tatctataca aaaaaatata gacaaactat atatataata tatttcaaat\n",
        "      781 ttccttatat atcctaatat aaaaatatct aataaattag atgaatatca aagaatctat\n",
        "      841 tgatttagtg tattattaaa tgtatatctt aattcaatat tattattcta ttcattttta\n",
        "      901 ttattcattt ttattcattt tcaaatttag aatatattaa tctatatatt aatttagaat\n",
        "      961 tctattctaa ttcgaattca atttttaaat attcatattc aattaaaatt gaaatttttt\n",
        "     1021 cattcgcgag gagccggatg agaagaaact ctcacgtccg gttctgtagt agaggtggaa\n",
        "     1081 ttaagaaaaa accatcaact ataaccccaa aagaaccaga ttctgtaaac aacatagagg\n",
        "     1141 aagaatgaag ggaatatctt atcgggggaa tcgtatttgt ttcggaagat atgctctcag\n",
        "     1201 cacga\n",
        "//\n",
        "\n",
        "LOCUS       HM041465                1190 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea dejecta ribosomal protein L16-like (rpl16) gene, partial\n",
        "            sequence; chloroplast.\n",
        "ACCESSION   HM041465\n",
        "VERSION     HM041465.1  GI:330887224\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia dejecta\n",
        "  ORGANISM  Opuntia dejecta\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1190)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1190)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1190\n",
        "                     /organism=\"Opuntia dejecta\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:1001110\"\n",
        "     gene            <1..>1190\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1096..>1190\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 tgatatacga aanagtaaga gcccatagta tgaagtatga actaataact atagaactaa\n",
        "       61 taaccaactc atcgcatcac attatctgga tccaaagaag cagtcaagat aggatatttt\n",
        "      121 ggtcctatca ttgcagcaac tgaatttttt tttcataaac aagaaatcaa atgagttgtc\n",
        "      181 aagcaaaaga aaaaaaaaaa aaaaaatata ctttaangga gggggatgcg gataaatgga\n",
        "      241 aaggcgaaag aaagaaaaaa atgaatctaa atgatatacg attccactat gtaaggtctt\n",
        "      301 tgaatcatat cataaaagac aatgtaataa agcatgaata cagattcaca cataattatc\n",
        "      361 tgtatgatct attcatagaa aaaagaaaaa agtaagagcc tccggccaat aaagactaag\n",
        "      421 agggttggct caagaacaaa gttcattaag ggctccattg tagaattcag acctaatcat\n",
        "      481 taatcaagaa gcgatgggaa cgatgtaatc catgaataca gaagattcaa ttgaaaaaga\n",
        "      541 atcctaatga ttcattggga aggatggcgg aacgaaccag agaccaattc atctattctg\n",
        "      601 aaaagtgata aactaatcct ataaaactaa aatagatatt gaaagagtaa atattcgccc\n",
        "      661 gcgaaaattc cttttttatt aaattgctca cattttattt tagcaatgca atctaataaa\n",
        "      721 atatatctat acaaaaaaat atagacaaac tatatatata atatatttca aatttcctta\n",
        "      781 tatatcctaa tataaaaata tctaataaat tagatgaata tcaaagaatc tattgattta\n",
        "      841 gtgtattatt aaatgtatat cttaattcaa tattattatt ctattcattt ttattattca\n",
        "      901 tttttattca ttttcaaatt tagaatatat taatctatat attaatttag aattctattc\n",
        "      961 taattcgaat tcaattttta aatattcata ttcaattaaa attgaaattt tttcattcgc\n",
        "     1021 gaggagccgg atgagaagaa actctcacgt ccggttctgt agtagaggtg gaattaagaa\n",
        "     1081 aaaaccatca actataaccc caaaagaacc agattctgta aacaacatag aggaagaatg\n",
        "     1141 aagggaatat cttatcgggg gaatcgtatt tgtttcggaa gatatgctct\n",
        "//\n",
        "\n",
        "LOCUS       HM041464                1184 bp    DNA     linear   PLN 03-MAY-2011\n",
        "DEFINITION  Nopalea cochenillifera ribosomal protein L16-like (rpl16) gene,\n",
        "            partial sequence; chloroplast.\n",
        "ACCESSION   HM041464\n",
        "VERSION     HM041464.1  GI:330887223\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia cochenillifera\n",
        "  ORGANISM  Opuntia cochenillifera\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia; Nopalea.\n",
        "REFERENCE   1  (bases 1 to 1184)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Hernandez,H.M., De-Nova,J.A., Puente,R.,\n",
        "            Eguiarte,L.E. and Magallon,S.\n",
        "  TITLE     Phylogenetic relationships and evolution of growth form in\n",
        "            Cactaceae (Caryophyllales, Eudicotyledoneae)\n",
        "  JOURNAL   Am. J. Bot. 98 (1), 44-61 (2011)\n",
        "   PUBMED   21613084\n",
        "REFERENCE   2  (bases 1 to 1184)\n",
        "  AUTHORS   Hernandez-Hernandez,T., Magallon,S.A., Hernandez,H.M., De-Nova,A.,\n",
        "            Puente,R. and Eguiarte,L.E.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (17-MAR-2010) Departamento de Botanica, Instituto de\n",
        "            Biologia, Universidad Nacional Autonoma de Mexico, 3er Circuito de\n",
        "            Ciudad Universitaria, Ciudad Universitaria, Coyoacan, Distrito\n",
        "            Federal C.P. 04510, Mexico\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..1184\n",
        "                     /organism=\"Opuntia cochenillifera\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:338184\"\n",
        "     gene            <1..>1184\n",
        "                     /gene=\"rpl16\"\n",
        "     misc_feature    <1102..>1184\n",
        "                     /gene=\"rpl16\"\n",
        "                     /note=\"similar to ribosomal protein L16\"\n",
        "ORIGIN      \n",
        "        1 gctgtgatat acgaaaaagt aagagcccat agtatgaagt atgaactaac aactatagaa\n",
        "       61 ctaataacca actcatcgca tcacattatc tggatccaaa gaagcagtca agataggata\n",
        "      121 ttttggtcct atcattgcag caactgaatt tttttttcat aaacaagaaa tcaaatgagt\n",
        "      181 tgtcaagcaa aagaaaaaaa aaaaaaaaaa tatactttaa aggaggggga tgcggataaa\n",
        "      241 tggaaaggcg aaagaaagaa aaaaatgaat ctaaatgata tacgattcca ctatgtaagg\n",
        "      301 tctttgaatc atatcataaa agacaatgta ataaagcatg aatacagatt cccacataat\n",
        "      361 tatctgatat gaatctattc atagaaaaaa gaaaaaagta agagcctccg gccaataaag\n",
        "      421 actaagaggg ttggctcaag aacaaagttc attaagggct ccattgtaga attcagacct\n",
        "      481 aatcattaat caagaagcga tgggaacgat gtaatccatg aatacagaag attcaattga\n",
        "      541 aaaagaatcc taatgattca ttgggaagga tggcggaacg aaccagagac caattcatct\n",
        "      601 attctgaaaa gtgataaact aatcctataa aactaaaata gatattgaaa gagtaaatat\n",
        "      661 tcgcccgcga aaattccttt tttattaaat tgctcacatt ttattttagc aatgcaatct\n",
        "      721 aataaaatat atctatacaa aaaaatatag acaaactata tatataatat atttcaaatt\n",
        "      781 tccttatata tcctaatata aaaatatcta ataaattaga tgaatatcaa agaatctatt\n",
        "      841 gatttagtgt attattaaat gtatatctta attcaatatt attattctat tcatttttat\n",
        "      901 tattcatttt tattcatttt caaatttaga atatattaat ctatatatta atttagaatt\n",
        "      961 ctattctaat tcgaattcaa tttttaaata ttcatattca attaaaattg aaattttttc\n",
        "     1021 attcgcgagg agccggatga gaagaaactc tcacgtccgg ttctgtagta gaggtggaat\n",
        "     1081 taagaaaaaa ccatcaacta taaccccaaa agaaccagat tctgtaaaca acatagagga\n",
        "     1141 agaatgaagg gaatatctta tcgggggaat cgtatttgtt tcgg\n",
        "//\n",
        "\n",
        "LOCUS       AY851612                 892 bp    DNA     linear   PLN 10-APR-2007\n",
        "DEFINITION  Opuntia subulata rpl16 gene, intron; chloroplast.\n",
        "ACCESSION   AY851612\n",
        "VERSION     AY851612.1  GI:57240072\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Austrocylindropuntia subulata\n",
        "  ORGANISM  Austrocylindropuntia subulata\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Austrocylindropuntia.\n",
        "REFERENCE   1  (bases 1 to 892)\n",
        "  AUTHORS   Butterworth,C.A. and Wallace,R.S.\n",
        "  TITLE     Molecular Phylogenetics of the Leafy Cactus Genus Pereskia\n",
        "            (Cactaceae)\n",
        "  JOURNAL   Syst. Bot. 30 (4), 800-808 (2005)\n",
        "REFERENCE   2  (bases 1 to 892)\n",
        "  AUTHORS   Butterworth,C.A. and Wallace,R.S.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (10-DEC-2004) Desert Botanical Garden, 1201 North Galvin\n",
        "            Parkway, Phoenix, AZ 85008, USA\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..892\n",
        "                     /organism=\"Austrocylindropuntia subulata\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:106982\"\n",
        "     gene            <1..>892\n",
        "                     /gene=\"rpl16\"\n",
        "     intron          <1..>892\n",
        "                     /gene=\"rpl16\"\n",
        "ORIGIN      \n",
        "        1 cattaaagaa gggggatgcg gataaatgga aaggcgaaag aaagaaaaaa atgaatctaa\n",
        "       61 atgatatacg attccactat gtaaggtctt tgaatcatat cataaaagac aatgtaataa\n",
        "      121 agcatgaata cagattcaca cataattatc tgatatgaat ctattcatag aaaaaagaaa\n",
        "      181 aaagtaagag cctccggcca ataaagacta agagggttgg ctcaagaaca aagttcatta\n",
        "      241 agagctccat tgtagaattc agacctaatc attaatcaag aagcgatggg aacgatgtaa\n",
        "      301 tccatgaata cagaagattc aattgaaaaa gatcctaatg atcattggga aggatggcgg\n",
        "      361 aacgaaccag agaccaattc atctattctg aaaagtgata aactaatcct ataaaactaa\n",
        "      421 aatagatatt gaaagagtaa atattcgccc gcgaaaattc cttttttatt aaattgctca\n",
        "      481 tattttattt tagcaatgca atctaataaa atatatctat acaaaaaaat atagacaaac\n",
        "      541 tatatatata taatatattt caaatttcct tatataccca aatataaaaa tatctaataa\n",
        "      601 attagatgaa tatcaaagaa tctattgatt tagtgtatta ttaaatgtat atcttaattc\n",
        "      661 aatattatta ttctattcat ttttattcat tttcaaattt ataatatatt aatctatata\n",
        "      721 ttaatttata attctattct aattcgaatt caatttttaa atattcatat tcaattaaaa\n",
        "      781 ttgaaatttt ttcattcgcg aggagccgga tgagaagaaa ctctcatgtc cggttctgta\n",
        "      841 gtagagatgg aattaagaaa aaaccatcaa ctataacccc aagagaacca ga\n",
        "//\n",
        "\n",
        "LOCUS       AY851611                 881 bp    DNA     linear   PLN 10-APR-2007\n",
        "DEFINITION  Opuntia polyacantha rpl16 gene, intron; chloroplast.\n",
        "ACCESSION   AY851611\n",
        "VERSION     AY851611.1  GI:57240071\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Opuntia polyacantha\n",
        "  ORGANISM  Opuntia polyacantha\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Opuntia.\n",
        "REFERENCE   1  (bases 1 to 881)\n",
        "  AUTHORS   Butterworth,C.A. and Wallace,R.S.\n",
        "  TITLE     Molecular Phylogenetics of the Leafy Cactus Genus Pereskia\n",
        "            (Cactaceae)\n",
        "  JOURNAL   Syst. Bot. 30 (4), 800-808 (2005)\n",
        "REFERENCE   2  (bases 1 to 881)\n",
        "  AUTHORS   Butterworth,C.A. and Wallace,R.S.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (10-DEC-2004) Desert Botanical Garden, 1201 North Galvin\n",
        "            Parkway, Phoenix, AZ 85008, USA\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..881\n",
        "                     /organism=\"Opuntia polyacantha\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:307728\"\n",
        "     gene            <1..>881\n",
        "                     /gene=\"rpl16\"\n",
        "     intron          <1..>881\n",
        "                     /gene=\"rpl16\"\n",
        "ORIGIN      \n",
        "        1 cattaaagga gggggatgcg gataaatgga aaggcgaaag aaagaaaaaa atgaatctaa\n",
        "       61 atgatatacg attccactat gtaaggtctt tgaatcatat cataaaagac aatgtaataa\n",
        "      121 agcatgaata cagattcaca cataattatc tgatatgaat ctattcatag aaaaaagaaa\n",
        "      181 aaagtaagag cctccggcca ataaagacta agagggttgg ctcaagaaca aagttcatta\n",
        "      241 agggctccat tgtagaattc agacctaatc attaatcaag aagcgatggg aacgatgtaa\n",
        "      301 tccatgaata cagaagattc aattgaaaaa gatcctaatg atcattggga aggatggcgg\n",
        "      361 aacgaaccag agaccaattc atctattctg aaaagtgata aactaatcct ataaaactaa\n",
        "      421 aatagatatt gaaagagtaa atattcgccc gcgaaaattc cttttttatt aaattgctca\n",
        "      481 cattttattt tagcaatgca atctaataaa atatatctat acaaaaaaat atagacaaac\n",
        "      541 tctatatata atatatttca aatttcctta tatatcctaa tataaaaata tctaataaat\n",
        "      601 tagatgaata tcaaagaatc tattgattta gtgtattatt aaatgtatat cttaattcaa\n",
        "      661 tattattatt ctattcattt tcaaatttag aatatattaa tctatatatt aatttagaat\n",
        "      721 tctattctaa ttcgaattca atttttaaat attcatattc aattaaaatt gaaatttttt\n",
        "      781 cattcgcgag gagccggatg agaagaaact ctcacgtccg gttactgtag tagaggtgga\n",
        "      841 attaagaaaa aaccatcaac tataacccca aaagaaccag a\n",
        "//\n",
        "\n",
        "LOCUS       AF191661                 895 bp    DNA     linear   PLN 07-NOV-1999\n",
        "DEFINITION  Opuntia kuehnrichiana rpl16 gene; chloroplast gene for chloroplast\n",
        "            product, partial intron sequence.\n",
        "ACCESSION   AF191661\n",
        "VERSION     AF191661.1  GI:6273287\n",
        "KEYWORDS    .\n",
        "SOURCE      chloroplast Cumulopuntia sphaerica\n",
        "  ORGANISM  Cumulopuntia sphaerica\n",
        "            Eukaryota; Viridiplantae; Streptophyta; Embryophyta; Tracheophyta;\n",
        "            Spermatophyta; Magnoliophyta; eudicotyledons; Gunneridae;\n",
        "            Pentapetalae; Caryophyllales; Cactineae; Cactaceae; Opuntioideae;\n",
        "            Cumulopuntia.\n",
        "REFERENCE   1  (bases 1 to 895)\n",
        "  AUTHORS   Dickie,S.L. and Wallace,R.S.\n",
        "  TITLE     Phylogeny of the subfamily Opuntioideae (Cactaceae)\n",
        "  JOURNAL   Unpublished\n",
        "REFERENCE   2  (bases 1 to 895)\n",
        "  AUTHORS   Dickie,S.L. and Wallace,R.S.\n",
        "  TITLE     Direct Submission\n",
        "  JOURNAL   Submitted (28-SEP-1999) Botany, Iowa State University, 353 Bessey\n",
        "            Hall, Ames, IA 50011-1020, USA\n",
        "FEATURES             Location/Qualifiers\n",
        "     source          1..895\n",
        "                     /organism=\"Cumulopuntia sphaerica\"\n",
        "                     /organelle=\"plastid:chloroplast\"\n",
        "                     /mol_type=\"genomic DNA\"\n",
        "                     /db_xref=\"taxon:106979\"\n",
        "                     /note=\"subfamily Opuntioideae; synonym: Cumulopuntia\n",
        "                     kuenrichiana\"\n",
        "     gene            <1..>895\n",
        "                     /gene=\"rpl16\"\n",
        "     intron          <1..>895\n",
        "                     /gene=\"rpl16\"\n",
        "ORIGIN      \n",
        "        1 tatacattaa agaaggggga tgcggataaa tggaaaggcg aaagaaagaa aaaaatgaat\n",
        "       61 ctaaatgata tacgattcca ctatgtaagg tctttgaatc atatcataaa agacaatgta\n",
        "      121 ataaagcatg aatacagatt cacacataat tatctgatat gaatctattc atagaaaaaa\n",
        "      181 gaaaaaagta agagcctccg gccaataaag actaagaggg ttggctcaag aacaaagttc\n",
        "      241 attaagagct ccattgtaga attcagacct aatcattaat caagaagcga tgggaacgat\n",
        "      301 gtaatccatg aatacagaag attcaattga aaaagatcct atgatccatt gggaaggatg\n",
        "      361 gcggaacgaa ccagagacca attcatctat tctgaaaagt gataaactaa tcctataaaa\n",
        "      421 ctaaaataga tattgaaaga gtaaatattc gcccgcgaaa attccttttt tttttaaatt\n",
        "      481 gctcatattt tattttagca atgcaatcta ataaaatata tctatacaaa aaaataaaga\n",
        "      541 caaactatat atataatata tttcaaattt ccttatatat ccaaatataa aaatatctaa\n",
        "      601 taaattagat gaatatcaaa gaatctattg atttagtgta ttattaaatg tatatcttaa\n",
        "      661 ttcaatatta ttattctatt catttttatt cattttcaat tttataatat attaatctat\n",
        "      721 atattaattt ataattctat tctaattcga attcaatttt taaatattca tattcaatta\n",
        "      781 aaattgaaat tttttcattc gcgaggagcc ggatgagaag aaactctcat gtccggttct\n",
        "      841 gtagtagaga tggaattaag aaaaaaccat caactataac cccaagagaa ccaga\n",
        "//\n",
        "\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 82
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In this case, we are just getting the raw records. To get the records in a more Python-friendly form, we can use `Bio.SeqIO` to parse the GenBank data into `SeqRecord` objects, including `SeqFeature` objects (see Chapter [5](http://biopython.org/DIST/docs/tutorial/Tutorial.html#chapter%3ABio.SeqIO)):"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import SeqIO\n",
      "handle4 = Entrez.efetch(db=\"nuccore\", id=gi_list, rettype=\"gb\", retmode=\"text\")\n",
      "result4 = SeqIO.parse(handle4, \"gb\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 83
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can now step through the records and look at the information we are interested in:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "for record in result4: \n",
      "    print(\"%s, length %i, with %i features\" % (record.name, len(record), len(record.features)))\n",
      "handle4.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "HQ621368, length 399, with 3 features\n",
        "HM041482, length 1197, with 3 features\n",
        "HM041481, length 1200, with 3 features"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "HM041480, length 1153, with 3 features\n",
        "HM041479, length 1197, with 3 features\n",
        "HM041478, length 1187, with 3 features\n",
        "HM041477, length 1197, with 3 features\n",
        "HM041476, length 1205, with 3 features"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "HM041474, length 1163, with 3 features\n",
        "HM041473, length 1203, with 3 features\n",
        "HM041472, length 1182, with 3 features\n",
        "HM041469, length 1189, with 3 features\n",
        "HM041468, length 1202, with 3 features"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "HM041467, length 1199, with 3 features\n",
        "HM041466, length 1205, with 3 features\n",
        "HM041465, length 1190, with 3 features\n",
        "HM041464, length 1184, with 3 features\n",
        "AY851612, length 892, with 3 features\n",
        "AY851611, length 881, with 3 features\n",
        "AF191661, length 895, with 3 features\n"
       ]
      }
     ],
     "prompt_number": 84
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using these automated query retrieval functionality is a big plus over doing things by hand. Although the module should obey the NCBI\u2019s max three queries per second rule, the NCBI have other recommendations like avoiding peak hours. See Section [9.1](#sec%3Aentrez-guidelines). In particular, please note that for simplicity, this example does not use the WebEnv history feature. You should use this for any non-trivial search and download work, see Section [9.15](#sec%3Aentrez-webenv).\n",
      "\n",
      "Finally, if plan to repeat your analysis, rather than downloading the files from the NCBI and parsing them immediately (as shown in this example), you should just download the records once and save them to a local file on your hard disk, and then parse the local file."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.14.4  Finding the lineage of an organism"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Staying with a plant example, let\u2019s now find the lineage of the _Cypripedioideae_ orchid family. First, we search the Taxonomy database for `\"Cypripedioideae\"`, which yields exactly one NCBI taxonomy identifier:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.esearch(db=\"Taxonomy\", term=\"Cypripedioideae\")\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "result[\"IdList\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 85,
       "text": [
        "['158330']"
       ]
      }
     ],
     "prompt_number": 85
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next, we use EFetch to retrieve this entry from the Taxonomy database."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle2 = Entrez.efetch(db=\"Taxonomy\", id=\"158330\", retmode=\"xml\")\n",
      "result2 = Entrez.read(handle2)\n",
      "handle2.close()\n",
      "result2[0].keys() # Note: efetch always returns a list of entries"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 86,
       "text": [
        "[u'Lineage',\n",
        " u'Division',\n",
        " u'ParentTaxId',\n",
        " u'PubDate',\n",
        " u'LineageEx',\n",
        " u'CreateDate',\n",
        " u'TaxId',\n",
        " u'Rank',\n",
        " u'GeneticCode',\n",
        " u'ScientificName',\n",
        " u'MitoGeneticCode',\n",
        " u'UpdateDate']"
       ]
      }
     ],
     "prompt_number": 86
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can get the lineage directly from this record:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "result2[0][\"Lineage\"]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 87,
       "text": [
        "'cellular organisms; Eukaryota; Viridiplantae; Streptophyta; Streptophytina; Embryophyta; Tracheophyta; Euphyllophyta; Spermatophyta; Magnoliophyta; Mesangiospermae; Liliopsida; Petrosaviidae; Asparagales; Orchidaceae'"
       ]
      }
     ],
     "prompt_number": 87
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The record data contains much more than just the information shown here - for example look under `\"LineageEx\"` instead of `\"Lineage\"` and you\u2019ll get the NCBI taxon identifiers of the lineage entries too."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aentrez-webenv\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "9.15  Using the history and WebEnv"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Often you will want to make a series of linked queries. Most typically, running a search, perhaps refining the search, and then retrieving detailed search results. You can do this by making a series of separate calls to Entrez. However, the NCBI prefer you to take advantage of the [Entrez History server](http://www.ncbi.nlm.nih.gov/books/NBK1058/#eutils_esayers-2-3) - for example combining ESearch and EFetch.\n",
      "\n",
      "Another typical use of the history server would be to combine EPost and EFetch. You use EPost to upload a list of identifiers, which starts a new history session. You then download the records with EFetch by referring to the session (instead of the identifiers)."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.15.1  Searching for and downloading sequences using the history"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Suppose we want to search and download all the Opuntia rpl16 nucleotide sequences, and store them in a FASTA file. As shown in Section [9.14.3](#sec%3Aentrez-search-fetch-genbank), we can use ESearch to get a list of GI numbers, and then use EFetch to download them all.\n",
      "\n",
      "However, the preferred approach is to use ESearch with the history feature. This is a 2-step process:\n",
      "\n",
      "1. Use ESearch with the option `usehistory=\"y\"` to store the search results (UIDs) on the Entrez History server and return `WebEnv` and `QueryKey` values with which to access that history\n",
      "2. Use EFetch to retrieve the results by reference to `WebEnv` and `QueryKey` values rather than the UIDs"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "search_handle = Entrez.esearch(db=\"nucleotide\", term=\"Opuntia[orgn] and rpl16\", usehistory=\"y\")\n",
      "search_result = Entrez.read(search_handle)\n",
      "search_handle.close()\n",
      "search_result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 88,
       "text": [
        "{u'Count': '17', u'RetMax': '17', u'IdList': ['377581039', '330887240', '330887239', '330887238', '330887237', '330887235', '330887233', '330887232', '330887231', '330887228', '330887227', '330887226', '330887225', '330887224', '330887223', '57240071', '6273286'], u'WarningList': {u'OutputMessage': [], u'PhraseIgnored': ['and'], u'QuotedPhraseNotFound': []}, u'TranslationStack': [{u'Count': '1617', u'Field': 'Organism', u'Term': '\"Opuntia\"[Organism]', u'Explode': 'Y'}, {u'Count': '21690', u'Field': 'All Fields', u'Term': 'rpl16[All Fields]', u'Explode': 'N'}, 'AND'], u'QueryTranslation': '\"Opuntia\"[Organism] AND rpl16[All Fields]', u'TranslationSet': [{u'To': '\"Opuntia\"[Organism]', u'From': 'Opuntia[orgn]'}], u'RetStart': '0', u'QueryKey': '1', u'WebEnv': 'NCID_1_715506407_130.14.22.215_9001_1403790704_1422644949'}"
       ]
      }
     ],
     "prompt_number": 88
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To retrieve the results from this ESearch, we'll need the list of UIDs (in the `\"IdList\"`field) and the number of UIDs (in the `\"Count\"` field)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "gi_list = search_result[\"IdList\"]\n",
      "count = int(search_result[\"Count\"])\n",
      "count, len(gi_list)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 89,
       "text": [
        "(17, 17)"
       ]
      }
     ],
     "prompt_number": 89
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When you specify `usehistory=\"y\"`, the ESearch result includes two additional pieces of information: the `WebEnv` session cookie (a unique identifier), and the `QueryKey` (typically, `\"1\"`)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "webenv = search_result[\"WebEnv\"]\n",
      "query_key = search_result[\"QueryKey\"] \n",
      "webenv, query_key"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 90,
       "text": [
        "('NCID_1_715506407_130.14.22.215_9001_1403790704_1422644949', '1')"
       ]
      }
     ],
     "prompt_number": 90
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Having stored these values in variables `webenv` and `query_key` we can use them as parameters to EFetch instead of providing the GI numbers as UIDs.\n",
      "\n",
      "While for small searches you might be OK downloading everything at once, it is better to download in batches. You use the `retstart` and `retmax` parameters to specify which range of search results you want returned (starting entry using zero-based counting, and maximum number of results to return). For example,"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "\n",
      "batch_size = 3\n",
      "out_handle = open(\"orchid_rpl16.fasta\", \"w\")\n",
      "for start in range(0, count, batch_size):\n",
      "    end = min(count, start + batch_size)\n",
      "    print(\"Fetching records %i thru %i\" % (start + 1, end))\n",
      "    fetch_handle = Entrez.efetch(db=\"nucleotide\", rettype=\"fasta\", retmode=\"text\",\n",
      "                                 retstart=start, retmax=batch_size,\n",
      "                                 webenv=webenv, query_key=query_key)\n",
      "    data = fetch_handle.read()\n",
      "    fetch_handle.close()\n",
      "    out_handle.write(data)\n",
      "out_handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Fetching records 1 thru 3\n",
        "Fetching records 4 thru 6"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 7 thru 9"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 10 thru 12"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 13 thru 15"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 16 thru 17"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 91
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For illustrative purposes, this example downloaded the FASTA records in batches of three. Unless you are downloading genomes or chromosomes, you would normally pick a larger batch size."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.15.2  Searching for and downloading abstracts using the history"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here is another history example, searching for papers published in the last year about the _Opuntia_, and then downloading them into a file in MedLine format:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "\n",
      "# invoke ESearch, using history\n",
      "search_handle = Entrez.esearch(db=\"pubmed\", term=\"Opuntia[ORGN]\",\n",
      "                               reldate=365, datetype=\"pdat\",\n",
      "                               usehistory=\"y\")\n",
      "search_result = Entrez.read(search_handle)\n",
      "search_handle.close()\n",
      "\n",
      "# save the count and webenv & query_key referencing the ESearch history\n",
      "count = int(search_result[\"Count\"])\n",
      "webenv = search_result[\"WebEnv\"]\n",
      "query_key = search_result[\"QueryKey\"] \n",
      "print(\"Fetching %i records using webenv=%s, query_key=%s\" % (count, webenv, query_key))\n",
      "\n",
      "# invoke EFetch to retrieve count records, batch_size at a time, using the webenv & query_key values rather than the UIDs\n",
      "batch_size = 10\n",
      "out_handle = open(\"recent_orchid_papers.txt\", \"w\")\n",
      "for start in range(0, count, batch_size):\n",
      "    end = min(count, start + batch_size)\n",
      "    print(\"Fetching records %i thru %i\" % (start + 1, end))\n",
      "    fetch_handle = Entrez.efetch(db=\"pubmed\",\n",
      "                                 rettype=\"medline\", retmode=\"text\",\n",
      "                                 retstart=start, retmax=batch_size,\n",
      "                                 webenv=webenv,\n",
      "                                 query_key=query_key)\n",
      "    data = fetch_handle.read()\n",
      "    fetch_handle.close()\n",
      "    out_handle.write(data)\n",
      "out_handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Fetching 51 records using webenv=NCID_1_715506898_130.14.22.215_9001_1403790706_34310305, query_key=1\n",
        "Fetching records 1 thru 10\n",
        "Fetching records 11 thru 20"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 21 thru 30"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 31 thru 40"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 41 thru 50"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 51 thru 51"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 92
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As described in Section [9.12.1](#subsec%3Aentrez-and-medline), you can then use `Bio.Medline` to parse the saved records."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "<a id=\"sec%3Aelink-citations\"></a>"
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "9.15.3  Searching for citations"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In Section [9.7](#sec%3Aelink) we mentioned ELink can be used to search for citations of a given paper. Unfortunately this only covers journals indexed for PubMed Central (doing it for all the journals in PubMed would mean a lot more work for the NIH). Let\u2019s try this for the Biopython PDB parser paper, PubMed ID 14630660:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pmid = \"14630660\"\n",
      "\n",
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle = Entrez.elink(dbfrom=\"pubmed\", db=\"pmc\", LinkName=\"pubmed_pmc_refs\", from_uid=pmid)\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "pmc_ids = [link[\"Id\"] for link in result[0][\"LinkSetDb\"][0][\"Link\"]]\n",
      "len(pmc_ids), pmc_ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 93,
       "text": [
        "(28,\n",
        " ['3879085',\n",
        "  '3661355',\n",
        "  '3531324',\n",
        "  '3461403',\n",
        "  '3394275',\n",
        "  '3394243',\n",
        "  '3312550',\n",
        "  '3253748',\n",
        "  '3161047',\n",
        "  '3144279',\n",
        "  '3122320',\n",
        "  '3102222',\n",
        "  '3096039',\n",
        "  '3057020',\n",
        "  '3036045',\n",
        "  '2944279',\n",
        "  '2902450',\n",
        "  '2744707',\n",
        "  '2705363',\n",
        "  '2682512',\n",
        "  '2483496',\n",
        "  '2447749',\n",
        "  '2098836',\n",
        "  '2072961',\n",
        "  '1933154',\n",
        "  '1848001',\n",
        "  '1192790',\n",
        "  '1190160'])"
       ]
      }
     ],
     "prompt_number": 93
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "'19304878' in pmc_ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 94,
       "text": [
        "False"
       ]
      }
     ],
     "prompt_number": 94
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "This returns the IDs of several articles, but the PubMed ID for our Biopython application note (19304878) has not been found. That is because these are not _PubMed_ IDs (_PMIDs_), but _PubMed Central_ IDs (_PMCIDs_). The PMCID for our application note (2682512) is in that list."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "'2682512' in pmc_ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 95,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 95
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "So, what if (like me) you\u2019d rather get back a list of PubMed IDs? Well we can call ELink again to translate them. This becomes a two step process, so by now you should expect to use the history feature to accomplish it.\n",
      "\n",
      "First, we'll do this without using the history feature, making a second (separate) call to ELink:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pmid2 = \"19304878\"\n",
      "\n",
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle2 = Entrez.elink(dbfrom=\"pmc\", db=\"pubmed\", LinkName=\"pmc_pubmed\", from_uid=\",\".join(pmc_ids))\n",
      "result2 = Entrez.read(handle2)\n",
      "handle2.close()\n",
      "pubmed_ids = [link[\"Id\"] for link in result2[0][\"LinkSetDb\"][0][\"Link\"]]\n",
      "print(\"Returned %i pubmed ids\" % len(pubmed_ids))\n",
      "print(\"Position of %s: %i\" % (pmid2, pubmed_ids.index(pmid2) if pmid2 in pubmed_ids else -1))\n",
      "pubmed_ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Returned 28 pubmed ids\n",
        "Position of 19304878: 19\n"
       ]
      },
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 96,
       "text": [
        "['24267035',\n",
        " '23717802',\n",
        " '23300419',\n",
        " '22784991',\n",
        " '22564897',\n",
        " '22553365',\n",
        " '22479120',\n",
        " '22361291',\n",
        " '21801404',\n",
        " '21637529',\n",
        " '21521828',\n",
        " '21500218',\n",
        " '21471012',\n",
        " '21467571',\n",
        " '20813068',\n",
        " '20607693',\n",
        " '20525384',\n",
        " '19698094',\n",
        " '19450287',\n",
        " '19304878',\n",
        " '18645234',\n",
        " '18502776',\n",
        " '18052533',\n",
        " '17888163',\n",
        " '17567620',\n",
        " '17397254',\n",
        " '16095538',\n",
        " '15985178']"
       ]
      }
     ],
     "prompt_number": 96
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The PubMed ID for our Biopython application note (19304878) is found in this list.\n",
      "\n",
      "Now, let\u2019s do that all again but with the history...TODO."
     ]
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Update: Searching for citations using ELink, EPost and EFetch _with history_"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "To use the history server with ELink (vs. ESearch), we will follow a 3-step process:\n",
      "\n",
      "1. Use ELink to find the UIDs of interest (in this case, PMCIDs)\n",
      "2. Use EPost to upload those UIDs to the Entrez History server, which returns `WebEnv` and `QueryKey` values with which to access that history\n",
      "3. Use EFetch to retrieve the results by reference to `WebEnv` and `QueryKey` values rather than the UIDs"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Note: this is the same first step as in the non-history version above\n",
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "pmid = \"14630660\"\n",
      "handle = Entrez.elink(dbfrom=\"pubmed\", db=\"pmc\", LinkName=\"pubmed_pmc_refs\", from_uid=pmid)\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "pmc_ids = [link[\"Id\"] for link in result[0][\"LinkSetDb\"][0][\"Link\"]]\n",
      "pmc_ids"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 97,
       "text": [
        "['3879085',\n",
        " '3661355',\n",
        " '3531324',\n",
        " '3461403',\n",
        " '3394275',\n",
        " '3394243',\n",
        " '3312550',\n",
        " '3253748',\n",
        " '3161047',\n",
        " '3144279',\n",
        " '3122320',\n",
        " '3102222',\n",
        " '3096039',\n",
        " '3057020',\n",
        " '3036045',\n",
        " '2944279',\n",
        " '2902450',\n",
        " '2744707',\n",
        " '2705363',\n",
        " '2682512',\n",
        " '2483496',\n",
        " '2447749',\n",
        " '2098836',\n",
        " '2072961',\n",
        " '1933154',\n",
        " '1848001',\n",
        " '1192790',\n",
        " '1190160']"
       ]
      }
     ],
     "prompt_number": 97
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "handle2 = Entrez.epost(db=\"pubmed\", id=\",\".join(pmc_ids))\n",
      "result2 = Entrez.read(handle2)\n",
      "handle2.close()\n",
      "result2"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 98,
       "text": [
        "{u'QueryKey': '1', u'WebEnv': 'NCID_1_640787000_130.14.18.34_9001_1403790713_2138330702'}"
       ]
      }
     ],
     "prompt_number": 98
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As before, we'll save the `WebEnv` and `QueryKey` values."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "webenv = result2[\"WebEnv\"]\n",
      "query_key = result2[\"QueryKey\"]\n",
      "webenv, query_key"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 99,
       "text": [
        "('NCID_1_640787000_130.14.18.34_9001_1403790713_2138330702', '1')"
       ]
      }
     ],
     "prompt_number": 99
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "However, unlike the earlier examples, the result returned by EPost does not include a `\"Count\"` field, so we must calculate that from the list of items we passed to EPost."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "count = len(pmc_ids)\n",
      "count"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 100,
       "text": [
        "28"
       ]
      }
     ],
     "prompt_number": 100
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We then use the `webenv` and `query_key` values with EFetch (rather than the list of UIDs) to retrieve the entries associated with the UIDs."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"     # Always tell NCBI who you are\n",
      "\n",
      "batch_size = 10\n",
      "out_handle = open(\"pmc_refs_%s.txt\" % pmid , \"w\")\n",
      "for start in range(0, count, batch_size):\n",
      "    end = min(count, start + batch_size)\n",
      "    print(\"Fetching records %i thru %i\" % (start + 1, end))\n",
      "    fetch_handle = Entrez.efetch(db=\"pubmed\",\n",
      "                                 rettype=\"medline\", retmode=\"text\",\n",
      "                                 retstart=start, retmax=batch_size,\n",
      "                                 webenv=webenv,\n",
      "                                 query_key=query_key)\n",
      "    data = fetch_handle.read()\n",
      "    fetch_handle.close()\n",
      "    out_handle.write(data)\n",
      "out_handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Fetching records 1 thru 10\n",
        "Fetching records 11 thru 20"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Fetching records 21 thru 28"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 101
    },
    {
     "cell_type": "heading",
     "level": 3,
     "metadata": {},
     "source": [
      "Update: Handling HTTPErrors"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "While working on solution to the use of ELink, EPost and EFetch above, I initially omitted the EPost step, and in doing so, encountered an [`HTTPError`](https://docs.python.org/2/library/urllib2.html#urllib2.HTTPError).\n",
      "\n",
      "The `Entrez` module includes an `_HTTPError` exception, but the underscore prefix and the following comment in `Entrez.__init__.py` suggests that this is not intended for general use:\n",
      "\n",
      "```\n",
      "#Importing these functions with leading underscore as not intended for reuse\n",
      "from Bio._py3k import urlopen as _urlopen\n",
      "from Bio._py3k import urlencode as _urlencode\n",
      "from Bio._py3k import HTTPError as _HTTPError\n",
      "```\n",
      "\n",
      "After asking about [`HTTPError` vs. `_HTTPError`](https://github.com/biopython/biopython/issues/343) as a Biopython Github issue, [peterjc](https://github.com/peterjc) explained:\n",
      "\n",
      ">If you want to catch an HTTPError from code in Bio.Entrez, we expect you to import this exception from the Python standard library, and not our local import of it.\n",
      "\n",
      ">i.e. Under Python 3:\n",
      "\n",
      ">`from urllib.error import HTTPError`\n",
      "\n",
      ">Or, under Python 2,\n",
      "\n",
      ">`from urllib2 import HTTPError`\n",
      "\n",
      "So I'm including the following example that demonstrates how to generate (and catch) an `HTTPError` exception ([\"HTTP 400: bad request\"](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1)). \n",
      "\n",
      "It is important to emphasize this is _not_ the proper way to use EUtils (or Entrez) - i.e., it omits the EPost step - but I am including this in case others similarly practice inadvertant impropriety."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "from urllib2 import HTTPError\n",
      "\n",
      "from Bio import Entrez\n",
      "Entrez.email = \"A.N.Other@example.com\"\n",
      "pmid = \"14630660\"\n",
      "# This first section is similar to the correct solution shown above, exept it uses the cmd=\"neighbor_history\" option in ELink\n",
      "handle = Entrez.elink(dbfrom=\"pubmed\", db=\"pmc\",\n",
      "                      LinkName=\"pubmed_pmc_refs\", from_uid=pmid,\n",
      "                      cmd=\"neighbor_history\") # this parameter is not correct (here); should use separate EPost instead\n",
      "result = Entrez.read(handle)\n",
      "handle.close()\n",
      "print(\"ELink result: %s\\n\" % result)\n",
      "\n",
      "# save the webenv & query_key: note, these require digging a bit deeper into the result returned by ELink\n",
      "webenv = result[0][\"WebEnv\"]\n",
      "query_key = result[0][\"LinkSetDbHistory\"][0][\"QueryKey\"]\n",
      "print(\"webenv=%s, query_key=%s\\n\" % (webenv, query_key))\n",
      "\n",
      "# Try using EFetch with an \"improper request\"\n",
      "try:\n",
      "    handle2 = Entrez.efetch(db=\"pubmed\",\n",
      "                            rettype=\"medline\", retmode=\"text\",\n",
      "                            retstart=0, retmax=1,\n",
      "                            webenv=webenv,\n",
      "                            query_key=query_key)\n",
      "except HTTPError as e: # using urllib2.HTTPError\n",
      "    print(e)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "ELink result: [{u'WebEnv': 'NCID_1_716200642_130.14.22.215_9001_1403794118_757400594', u'ERROR': [], u'DbFrom': 'pubmed', u'LinkSetDbHistory': [{u'DbTo': 'pmc', u'QueryKey': '1', u'LinkName': 'pubmed_pmc_refs'}], u'LinkSetDb': [], u'IdList': ['14630660']}]\n",
        "\n",
        "webenv=NCID_1_716200642_130.14.22.215_9001_1403794118_757400594, query_key=1\n",
        "\n",
        "HTTP Error 400: Bad Request"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 103
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}