Difference between revisions of "GMOD RPC API"

Revision as of 06:02, 16 January 2010

Document Status

In progress.

Background

This effort was started after Josh Goodman's talk at the January 2009 GMOD Meeting meeting titled "MOD Web API (A RESTful interface for MODs)". The main idea is to increase interoperability among the various model organism databases by creating an easy to use high level RESTful API. The queries iterated below are currently in the proposal stage and have no been implemented at any MOD.

Members

• Josh Goodman - FlyBase
• Robert Buels - Sol Genomics Network
• Richard Smith - InterMine
• Giles Velarde - GeneDB
• Todd Harris - WormBase
• Jim Hu - EcoliWiki
• Add your name here

Goals

• Data model agnostic
• Programming language agnostic
• Easy to use
• Versioned URLs for API stability over time

Related projects

• Common url
• GMOD
• Bio GMOD GenericGenePage

Data classes

At present, this API only covers querying and retrieving information for the gene data class.

API Version

In order to provide a stable URL API all web calls should be versioned according to what version of the GMOD REST API they are using. The version number included in the URLs corresponds to the API version and not the data version. As changes to the API are made the version number will be incremented. Access to the older API versions should be provided indefinitely.

Current GMOD REST API Version: 1.1

Data version

The search results must contain an XML tag called data_version that contains the database release number. If the database does not use release numbers then the date for when the entire data class was updated should be used. If that is not available or if individual records within a data class are updated and released asynchronously then a timestamp derived from the time of query execution should be used.

For example, FlyBase releases its data on a roughly monthly basis and tags all data contained in that release with a release number (e.g. FB2008_10). If a release number was not used then a date stamp for when the entire genes data class was updated should be used. If this is not available or if individual genes are updated in a piecemeal fashion then a timestamp based on the query execution time should be used.

Result dates

The individual search results must provide a creation and last modified timestamp via the date_created and last_modified XML tags respectively. If this information is not available then the current timestamp at query execution should be used.

Return types

Each service may define its own return types. The client may request a specific return type by appending the appropriate file extension to the URL. If no file extension is appended then the default return type for that service is used.

Compression

Sending of compressed XML and JSON results should follow the same rules used for HTTP communication. If a client wishes to receive compressed output it should indicate this by setting the Accept-Encoding HTTP request header. When a service provider receives a request it should check for the Accept-Encoding request header and compress the output if it supports the requested compression algorithm. If the service provider does compress the content it should then set the Content-Encoding HTTP response header to indicate the compression algorithm used.

Supported HTTP methods

Currently this API only describes read only services that use the GET HTTP method. There are no plans to provide create, update or delete operations at this time.

Querying or Filtering by Organism

Some of the queries that have been defined below allow for querying and/or filtering by organism by using Taxonomy IDs from the NCBI's Taxonomy database (http://www.ncbi.nlm.nih.gov/Taxonomy/) or the genus and/or species name (e.g. 'Drosophila melanogaster'). Non-leaf taxons should be supported by the service provider. For example, if you want to query all species in the Sophophora submitting the taxon ID 32341 should allow you to perform such a query. In order to implement this service you can either implement your own methods or utilize the NCBI's elink application that is part of their Eutils service (e.g. http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?id=32341&db=taxonomy&dbfrom=taxonomy)

Pretty URLs

In version 1.1 we have opted for a mixed approach of forming URLs that uses directory path structures to specify some elements and parameters to add additional filters.

Error handling

HTTP error code 400

An HTTP 400 status code should be returned for the following errors.

• not implemented - returned when a particular service is not implemented.
• invalid term - returned when a particular CV term is not used by the service.

Use Cases

• GMOD REST API Use Cases

Services

Searches

Organism List

Service Name

organisms

Purpose

Provides a list of organisms that are able to be queried with the service provider.

URL

http://yourmod.org/gmodrpc/v<api version>/organisms[.xml | .json]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/organisms
• http://flybase.org/gmodrpc/v1.1/organisms.xml
• http://flybase.org/gmodrpc/v1.1/organisms.json

XML Result

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <result>
     <organism>
        <genus>Drosophila</genus>
        <species>melanogaster</species>
        <taxonomy_id>7227</taxonomy_id>
     </organism>
  </result>
  <result>
     <organism>
         <species>Drosophila</species>
         <genus>simulans</genus>
         <taxonomy_id>7240</taxonomy_id>
     </organism>
   </result>

</resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       result:[
           {
               organism:{
                   genus:'Drosophila',
                   species:'melanogaster',
                   taxonomy_id:7227
               }
           },
           {
               organism:{
                   species:'Drosophila',
                   genus:'simulans',
                   taxonomy_id:7240
               }
           }
       ]
   }
}

Gene full text search

Service Name

fulltext

Purpose

Performs a full text search on records and returns the IDs for matching records.

Description

This service returns features that contain the search term anywhere in the gene record. Results can be restricted to a specific organism by supplying the NCBI taxonomy id.

URL

http://yourmod.org/gmodrpc/v<api version>/fulltext/<search term>[.xml | .json]?[type=<SO term ID>][taxid=<taxid>][genus=<genus>][species=<species>]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/fulltext/cotransfection?type=SO:0000704 - Find genes that contain the term cotransfection.
• http://flybase.org/gmodrpc/v1.1/fulltext/cotransfection?type=SO:0000704&taxid=7227 - Find Drosophila melanogaster genes that contain the term cotransfection.
• http://flybase.org/gmodrpc/v1.1/fulltext/AE003845.json?type=SO:0000704 - Find genes that contain the term AE003845 and return a JSON result.
• http://flybase.org/gmodrpc/v1.1/fulltext/IPR000483.json?taxid=7240&type=SO:0000704 - Find Drosophila simulans genes that are labeled with InterPro ID IPR000483.

XML Result

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <query_time>2009-01-15 09:03:00</query_time>
  <query_url>http://flybase.org/gmodrpc/v1.1/fulltext/cotransfection?type=SO:0000704</query_url>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0085432</accession>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085432</url>
     </dbxref>
     <date_created>2003-03-08 00:00:00</date_created>
     <last_modified>2005-01-15 09:03:00</last_modified>
  </result>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0004364</accession>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0004364</url>
     </dbxref>
     <date_created>2005-01-08 00:00:00</date_created>
     <last_modified>2009-01-01 00:00:00</last_modified>
  </result>

</resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       query_time:'2009-01-15 09:03:00',
       query_url:'http://flybase.org/gmodrpc/v1.1/fulltext/cotransfection?type=SO:0000704',
       result:[
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0085432',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085432'
               },
               date_created:'2003-03-08 00:00:00',
               last_modified:'2005-01-15 09:03:00'
           },
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0004364',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0004364'
               },
               date_created:'2005-01-08 00:00:00',
               last_modified:'2009-01-01 00:00:00'
           }
       ]
   }
}

Gene location

Service Name

location

Purpose

Retrieves a list of features that lie within a specific sequence range.

Description

This service returns features that lie within a specific sequence range. Arguments include the name of the largest assembled unit (i.e. chromosome, scaffold, etc...), location start (fmin), location stop (fmax), and strand. Results can be restricted to a specific organism by supplying the NCBI taxonomy id.

URL

http://yourmod.org/gmodrpc/v<api version>/location/chromosome/<name>[.xml | .json]?[type=<SO term ID>][fmin=<int>][fmax=<int>][strand=(1 | -1)][taxid=<taxid>][genus=<genus>][species=<species>]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/location/chromosome/X?type=SO:0000704 - Find all genes that are localized on the X chromosome of any Drosophila species.
• http://flybase.org/gmodrpc/v1.1/location/chromosome/X?type=SO:0000704&taxid=7227 - Find all genes that are localized to the X chromosome of melanogaster.
• http://flybase.org/gmodrpc/v1.1/location/chromosome/X.json?type=SO:0000704&fmin=40000&taxid=7227 - Find all genes that are localized on the X chromosome in the range of 40,000 bp to the end of X on any strand.
• http://flybase.org/gmodrpc/v1.1/location/chromosome/X.xml?type=SO:0000704&genus=Drosophila&species=melanogaster&fmax=40000 - Find all genes that are localized on the X chromosome in the range of the start of the X up to and including 40,000 bp on any strand.
• http://flybase.org/gmodrpc/v1.1/location/chromosome/X.xml?type=SO:0000704&fmin=50000&fmax=140000&strand=1&taxid=7227 - Find all genes that are localized on the plus strand of the X chromosome between and including 50,000 bp and 140,000 bp.

XML Result

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <query_time>2009-01-15 09:03:00</query_time>
  <query_url>http://flybase.org/gmodrpc/v1.1/location/chromosome/2L?type=SO:0000704&fmin=12587000&fmax=12629000&taxid=7227</query_url>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0085424</accession>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085424</url>
     </dbxref>
     <date_created>2003-03-08 00:00:00</date_created>
     <last_modified>2005-01-15 09:03:00</last_modified>
  </result>

</resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       query_time:'2009-01-15 09:03:00',
       query_url:'http://flybase.org/gmodrpc/v1.1/location/chromosome/2L?type=SO:0000704&fmin=12587000&fmax=12629000&taxid=7227',
       result:[
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0085424',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085424'
               },
               date_created:'2003-03-08 00:00:00',
               last_modified:'2005-01-15 09:03:00'
           }
       ]
   }
}

Gene Ontology search

Service Name

ontology

Purpose

Search for genes that have been annotated with a particular gene ontology ID.

Description

This service returns genes that have been annotated with a particular ontology term. Results can be restricted to a specific organism by supplying the NCBI taxonomy id.

URL

http://yourmod.org/gmodrpc/v<api version>/ontology/gene/<ontology ID>[.xml | .json]?[taxid=<taxid>][genus=<genus>][species=<species>]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/ontology/gene/GO:0005515 - Find all genes annotated with GO:0005515.
• http://flybase.org/gmodrpc/v1.1/ontology/gene/!GO:0005515 - Find all genes annotated with GO:0005515 and the GO NOT operator.
• http://flybase.org/gmodrpc/v1.1/ontology/gene/GO:0005515?taxid=7227 - Find all Drosophila melanogaster genes annotated with GO:0005515.
• http://flybase.org/gmodrpc/v1.1/ontology/gene/GO:0005515.json - Find all genes annotated with GO:0005515 and return a JSON result.

XML Result

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <query_time>2009-01-15 09:03:00</query_time>
  <query_url>http://flybase.org/gmodrpc/v1.1/ontology/gene/GO:0005515</query_url>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0085432</accession>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085432</url>
     </dbxref>
     <date_created>2003-03-08 00:00:00</date_created>
     <last_modified>2005-01-15 09:03:00</last_modified>
  </result>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0004364</accession>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0004364</url>
     </dbxref>
     <date_created>2005-01-08 00:00:00</date_created>
     <last_modified>2009-01-01 00:00:00</last_modified>
  </result>

</resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       query_time:'2009-01-15 09:03:00',
       query_url:'http://flybase.org/gmodrpc/v1.1/ontology/gene/GO:0005515',
       result:[
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0085432',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0085432'
               },
               date_created:'2003-03-08 00:00:00',
               last_modified:'2005-01-15 09:03:00'
           },
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0004364',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0004364'
               },
               date_created:'2005-01-08 00:00:00',
               last_modified:'2009-01-01 00:00:00'
           }
       ]
   }
}

Gene ortholog search

Service name

orthologs/gene

Purpose

Search for orthologs of the supplied gene ID.

Description

This service returns genes that have been determined by some means to be orthologous to the supplied gene ID. If the supplied gene ID is within the namespace of the web service provider then all known orthologs of that gene are returned. If the supplied gene ID is not within the namespace of the web service provider then it only returns genes for organisms that are offered by the service provider.

For example, for a given gene FlyBase stores orthology calls to other FlyBase genes and non FlyBase genes. Thus, given a FlyBase gene ID you can obtain a list of gene IDs for genes within FlyBase and to other non Drosophila species. In addition, given a non FlyBase gene ID you can obtain a list of FlyBase genes that are orthologous to it.

Results can be restricted to a specific organism by supplying the NCBI taxonomy ID.

URL

http://yourmod.org/gmodrpc/v<api version>/orthologs/gene/<gene ID>[.xml | .json]?[taxid=<taxid>][genus=<genus>][species=<species>]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/orthologs/gene/FBgn0004364 - Find all FlyBase and non FlyBase genes that are orthologous to FBgn0004364.
• http://flybase.org/gmodrpc/v1.1/orthologs/gene/FBgn0004364?taxid=7240 - Find out if FBgn0004364 has an ortholog in Drosophila simulans.
• http://flybase.org/gmodrpc/v1.1/orthologs/gene/WBGene12345.json - Find all FlyBase genes that are orthologous to WBGene12345 and return a JSON result.

XML Result:

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <query_time>2009-01-15 09:03:00</query_time>
  <query_url>http://flybase.org/gmodrpc/v1.1/orthologs/gene/FBgn0000490</query_url>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0097591</accession>
        <data_provider>FlyBase</data_provider>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0097591</url>
     </dbxref>
  </result>
  <result>
     <dbxref>
         <db>ENSEMBL</db>
         <accession>ENSBTAP00000004992</accession>
         <data_provider>InParanoid</data_provider>
     </dbxref>
  </result>

</resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       query_time:'2009-01-15 09:03:00',
       query_url:'http://flybase.org/gmodrpc/v1.1/orthologs/gene/FBgn0000490',
       result:[
           {
               dbxref:{
                   db:'FlyBase',
                   accession:'FBgn0097591',
                   data_provider:'FlyBase'
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0097591'
               }
           },
           {
               dbxref:{
                   db:'ENSEMBL',
                   accession:'ENSBTAP00000004992',
                   data_provider:'InParanoid'
               }
           }
       ]
   }
}

Organism ortholog search

Service name

orthologs/organism

Purpose

Returns a list of orthologs between an organism and one or more other organisms.

Description

This service returns genes that have been determined by some means to be orthologous between a single organism and one or more other organisms. The organisms are specified by providing taxonomy IDs.

URL

http://yourmod.org/gmodrpc/v<api version>/orthologs/organism[/<taxid>][/genus/<genus>/species/<species>][.xml | .json]?[to_taxid=<taxid>][to_genus=<genus>&to_species=<species>]

Return types

XML or JSON

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/orthologs/organism/genus/Drosophila/species/melanogaster - Find all orthologs between Dmel and any other species.
• http://flybase.org/gmodrpc/v1.1/orthologs/organism/7227.xml?to_taxid=7240 - Find all orthologs between Dmel and Dsim.
• http://flybase.org/gmodrpc/v1.1/orthologs/organism/7227.json?to_taxid=7240 - Same as above except returned in JSON format.

XML Result:

<xml> <?xml version="1.0" encoding="UTF-8"?> <resultset>

  <api_version>1.1</api_version>
  <data_provider>FlyBase</data_provider>
  <data_version>FB2008_10</data_version>
  <query_time>2009-01-15 09:03:00</query_time>
  <query_url>http://flybase.org/gmodrpc/v1.1/orthologs/organism/genus/Drosophila/species/melanogaster</query_url>
  <result>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0000490</accession>
        <taxonomy_id>7227</taxonomy_id>
        <data_provider>FlyBase</data_provider>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0000490</url>
     </dbxref>
     <dbxref>
        <db>FlyBase</db>
        <accession>FBgn0015673</accession>
        <taxonomy_id>7240</taxonomy_id>
        <data_provider>FlyBase</data_provider>
        <url>http://flybase.org/gmodrpc/v1.1/fetch/FBgn0015673</url>
     </dbxref>
     <dbxref>
        <db>WormBase</db>
        <accession>WBGene00006570</accession>
        <taxonomy_id>6239</taxonomy_id>
        <data_provider>InParanoid</data_provider>
        <url>http://wormbase.org/gmodrpc/v1.1/fetch/WBGene00006570</url>
     </dbxref>
  </result>

... </resultset> </xml>

JSON Result

{
   resultset:{
       api_version:1.1,
       data_provider:'FlyBase',
       data_version:'FB2008_10',
       query_time:'2009-01-15 09:03:00',
       query_url:'http://flybase.org/gmodrpc/v1.1/orthologs/organism/genus/Drosophila/species/melanogaster',
       result:[
         {
           dbxref:[
               {
                   db:'FlyBase',
                   accession:'FBgn0000490',
                   taxonomy_id:7227,
                   data_provider:'FlyBase',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0000490'
               },
               {
                   db:'FlyBase',
                   accession:'FBgn0015673'
                   taxonomy_id:7240,
                   data_provider:'FlyBase',
                   url:'http://flybase.org/gmodrpc/v1.1/fetch/FBgn0015673'
               },
               {
                   db:'WormBase',
                   accession:'WBGene00006570',
                   taxonomy_id:6239,
                   data_provider:'InParanoid',
                   url:'http://wormbase.org/gmodrpc/v1.1/fetch/WBGene00006570'
               }
           ]
         }
       ]
   }
}

Fetching

Gene records

Service name

fetch/gene

Purpose

To fetch gene records in the Generic gene page XML format as implemented by Bio GMOD GenericGenePage.

Description

URL

http://yourmod.org/gmodrpc/v<api version>/fetch/gene/<gene ID>

Return types

XML

Default return type

XML

Example URLs

• http://flybase.org/gmodrpc/v1.1/fetch/gene/FBgn0097591

XML Result

See Bio GMOD GenericGenePage for example XML.

Sequence

Service name

fetch/seq

Purpose

Returns FASTA sequence for the specified query.

Description

URL

http://yourmod.org/gmodrpc/v<api version>/fetch/seq/<ID>[/<SO term ID of subtype>]

Return types

FASTA

Example URLs

• http://flybase.org/gmodrpc/v1.1/fetch/seq/FBgn0097591.fasta - Fetch the FASTA sequence of FBgn0097591
• http://flybase.org/gmodrpc/v1.1/fetch/seq/FBgn0097591/SO:0000204.fasta - Download all 5' UTRs of the gene FBgn0097591
• http://flybase.org/gmodrpc/v1.1/fetch/seq/FBgn0097591/SO:0000316.fasta - Download all CDS features of the gene FBgn0097591
• http://flybase.org/gmodrpc/v1.1/fetch/seq/FBgn0097591/SO:0000316.fasta - Download all CDS features of the gene FBgn0097591

FASTA Result:

TODO

• WADLs for all services?