|
|
| (203 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | System overview and installation
| + | #REDIRECT [[BioMart Tutorial]] |
| − | | + | |
| − | Prerequisites for BioMart
| + | |
| − | | + | |
| − | Software: Java 1.6, Ant and SVN client
| + | |
| − | | + | |
| − | OS: major Linux distributions
| + | |
| − | | + | |
| − | Server: min. one GB memory, three GB for better performance
| + | |
| − | Download BioMart
| + | |
| − | | + | |
| − | Installation of BioMart has already been done on the VM image. Refer to (http://www.biomart.org/rc6_documentation.pdf) for installation guidelines.
| + | |
| − | | + | |
| − | Creating data mart using MartBuilder
| + | |
| − | | + | |
| − | This section will show you how to create a VEGA mart using MartBuilder.
| + | |
| − | Download [ftp://anonymous@ftp.ebi.ac.uk/pub/software/biomart/martj_current/martj-bin.tgz MartJ]
| + | |
| − | MartJ contains applications necessary to create a Mart: MarBuilder and MartRunner. Installation of MartJ has already been done on the VM image. Check Section 2 in [http://www.biomart.org/user-docs.pdf Installing BioMart]
| + | |
| − | | + | |
| − | Uncompress it with:
| + | |
| − | tar zxvf martj-bin.tgz
| + | |
| − | | + | |
| − | This will result in a ''martj'' directory being created under the same directory as the tarball.
| + | |
| − | | + | |
| − | Creating/loading sample mart
| + | |
| − | | + | |
| − | 1. Open MartBuilder:
| + | |
| − | from the newly created ''martj'' directory
| + | |
| − | bin/martbuilder.sh
| + | |
| − | | + | |
| − | You will see a blank mart, waiting to have one or more source schemata added to it from which datasets will be generated later. The next step is to connect to your existing relational database and add the schema which contains the data which you wish to transform.
| + | |
| − | | + | |
| − | 2. Choose ''Add'' from the ''Schema'' menu. Input connection parameters as follows:
| + | |
| − | | + | |
| − | [[File:Mb_add_schema2.png]]
| + | |
| − | | + | |
| − | The proper information for the Target database and Target schema fields
| + | |
| − | Note that if you did not specify a schema when creating your database, your tables will be in the default schema for your platform:
| + | |
| − | | + | |
| − | MySQL: does not have schema, will be the same as the database name
| + | |
| − | | + | |
| − | PostGreSQL: “public”
| + | |
| − | | + | |
| − | Oracle and DB2: the username of the user who created the database
| + | |
| − | | + | |
| − | SQL Server: “dbo”
| + | |
| − | | + | |
| − | 3. Create datasets
| + | |
| − | | + | |
| − | Now you should be able to see the VEGA schema.
| + | |
| − | | + | |
| − | Find the table that contains the data you wish to use in the main table(s) of your dataset. Right-click on the table, and choose Create dataset. Here we selected both Gene and Transcript tables to be the main tables.
| + | |
| − | | + | |
| − | Click Create. Mart will look like this:
| + | |
| − | | + | |
| − | | + | |
| − | Your dataset does not actually exist yet. In order to create it, you need to generate the SQL for it then execute that SQL against your database.
| + | |
| − | | + | |
| − | 4. We now going to transform the source data into target dataset, but before that, we have to create a target database:
| + | |
| − | | + | |
| − | mysql -ubiomart -pbiomart -e 'create database mini_hsap_vega_mart'
| + | |
| − | | + | |
| − | Also we have to have martrunner running. Lets run it over port 9876.
| + | |
| − | | + | |
| − | 5. Start martrunner with:
| + | |
| − | | + | |
| − | ./bin/martrunner.sh
| + | |
| − | | + | |
| − | MartBuilder will send the transformation SQL to MartRunner through port 9876, and MartRunner will execute the transformation SQL. Usually, MartBuilder and MartRunner run on different machines.
| + | |
| − | | + | |
| − | We go back to MartBuilder, and click on the BuildMart.
| + | |
| − | | + | |
| − | | + | |
| − | The proper information for the Target database and Target schema fields differ depending on the database server type:
| + | |
| − | | + | |
| − | MySQL-- Target database and target schema must be the same, and different than the original source database and schema. The database must exist on the server.
| + | |
| − | | + | |
| − | SQL Server -- Target database and target schema both must exist on the server. The original source schema should not be used.
| + | |
| − | | + | |
| − | PostgreSQL, Oracle, and DB2 -- Target database must be the same as the original source. The target schema should exist within this database, and should be different than the original source schema. Start job in MartRunner (windows should have poped up upon SQL generation):
| + | |
| − | | + | |
| − | 5. Monitor progress using the Monitor MartRunner progress
| + | |
| − | | + | |
| − | | + | |
| − | Ensure everything is successful (green font):
| + | |
| − | | + | |
| − | | + | |
| − | Your schema now contains a mart with a complete dataset ready for configuring with MartConfigurator.
| + | |
| − | Configuring Using MartConfigurator
| + | |
| − | | + | |
| − | This section will show you how to configure the created VEGA mart using MartConfigurator
| + | |
| − | | + | |
| − | Run MartConfigurator
| + | |
| − | | + | |
| − | Run MartConfigurator with the following command in the directory of your installation:
| + | |
| − | | + | |
| − | ./dist/scripts/martconfigurator.sh
| + | |
| − | | + | |
| − | Add Source
| + | |
| − | | + | |
| − | In the MartConfigurator window, click *Add Mart *in the upper right corner.
| + | |
| − | | + | |
| − | A list of available marts will appear in the left corner. Click mini_hsap_vega_mart, and click Add. The new mart will appear in the left-hand corner of the MartConfigurator window.
| + | |
| − | | + | |
| − | VIEW ??? Schema, management window??
| + | |
| − | | + | |
| − | Create and configure access point.
| + | |
| − | | + | |
| − | To create an access point click on the +Add Access Point button in the right-hand Portal section.
| + | |
| − | | + | |
| − | You will be given a list of the existing marts to choose which one you would like to make an access point for. After giving the new access point a name of your choice, it will appear in the GUI tab. Double-clicking on the access point will open a new window that allows you to modify the access point.
| + | |
| − | | + | |
| − | The display name of any object (a container, attribute, or filter) can be changed by selecting that object (by clicking on it) and then double clicking
| + | |
| − | the displayname property in the lower right-hand pane.
| + | |
| − | | + | |
| − | Deploy BioMart Server
| + | |
| − | | + | |
| − | To test the registry, you can now click on the Start Server button in the upper right. You will be prompted to save the registry file you have created, and after doing so, the server will be deployed on your local machine, port 9000. Your web page should open automatically when the local server is ready. You can also deploy BioMart from the command line on the server. To deploy BioMart, from the directory of your installation, run the following command:
| + | |
| − | | + | |
| − | ./dist/scripts/biomart-server.sh start
| + | |
| − | | + | |
| − | To Stop the server, use the command:
| + | |
| − | | + | |
| − | ./dist/scripts/biomart-server.sh stop
| + | |
| − | More exercises with MartConfigurator
| + | |
| − | | + | |
| − | Set Filter typesSet Attribute URLSelect GUI type
| + | |
| − | | + | |
| − | Creating Links between sources
| + | |
| − | | + | |
| − | If two data sources contain common information (e.g. a Gene/Protein ID), this can be used to create a link, allowing filters and attributes from one data source to appear in the other. These are called “pointer attributes” and “pointer filters,” and the attribute or filter to which they point is called the “target. ”To add a pointer to an access point, double click on that access point in the portal tab to edit it. In the top left corner of the editing window, click on the Import from sources button.
| + | |
| − | | + | |
| − | You will be given a list of the existing data sources to choose which one you would like to make an access point for. After giving the new access point a name of your choice, it will appear in the GUI tab.
| + | |
| − | Double-clicking on the access point icon will open a new window that allows you to modify the access point.
| + | |
| − | References
| + | |
| − | | + | |
| − | To read more about BioMart, refer to the recent articles describing the BioMart software and its applications.
| + | |
| − | | + | |
| − | 1. Zhang J, Haider S, Baran J, Cros A, Guberman JM, Hsu J, Liang Y, Yao L, Kasprzyk A. BioMart: a data federation framework for large collaborative projects. Database (Oxford). 2011 Sep 19:2011:bar038.
| + | |
| − | | + | |
| − | 2.Guberman JM, Ai J, Baran J., et al. BioMart Central Portal: An Open Database Network for the Biological Community. Database (Oxford). 2011 Sep18;2011:bar041.
| + | |
| − | | + | |
| − | 3. Zhang J, Baran J, Guberman JM, Haider, S, Hsu J, Liang Y, Rivkin E, Wang J, Whitty B, Wong-Erasmus M, Yao L, Kasprzyk A. International Cancer Genome Consortium Data Portal - a One-stop Shop for Cancer Genomics Data. Database (Oxford). 2011 Sep19;2011:bar026.
| + | |
| | | | |
| | [[Category:BioMart]] | | [[Category:BioMart]] |
