Computing enrivronment
From Shiu Lab
Contents |
[edit]
Computer usage guidelines
The use of lab computer is subject to the fairuse policy in the MSU Statement of Acceptable Use. In addition, the lab computer should not be used for:
- Peer-to-peer or any type of file transfer if the files are not work related.
- Assessing sites with indecent content, such as pornagraphy.
- Gaming.
[edit]
Ergonomics
- We spend too much time on computers and this prolonged use generally results in discomfort or even injuries over time. Below is some info on the ergonomic consideration and exercise you can do.
[edit]
Software
- The use of open source software is encouraged!
- InkScape
- Inkscape is an Open Source vector graphics editor, with capabilities similar to Illustrator, Freehand, CorelDraw, or Xara X using the W3C standard Scalable Vector Graphics (SVG) file format.
- OpenOffice
- OpenOffice.org is a multiplatform and multilingual office suite and an open-source project. Compatible with all other major office suites, the product is free to download, use, and distribute.
- Cygwin
- Cygwin is a Linux-like environment for Windows.
[edit]
Printer and wireless setup
- Lab printer setup
- We have a black-and-white laserjet capable of double-sided printing. Please print double-sided when possible. Below is the info you need for setting up the printer.
Printer model: HP Laserjet 1320n IP address: 35.15.76.67
- Poster printer
- The Department has a poster printer available to us with reasonable cost. Check out the HP DesignJet 800 (Poster Printer) Policy.
- Wireless access
- The wireless access point is password protected.
Access point: Bart Password: go ask Shinhan
[edit]
Learn programming
[edit]
Python
- The PI's favorite language is Python. For those never program before, there are plenty of online tutorials for python. I'd suggest the official one to start. Also check out Dive into Python, a fantastic FREE online python programming book, also available in print form. Another really good introductory python book is Learning Python. Go through the first several chapters and you will be ready to roam!
- Python style guide
[edit]
R
- The R project consists of a comprehensive collection of statistical packages, including those for microarray data analyses.
- There are a bunch of manuals for R. But for beginner, look at the tutorial, Introduction to R.
[edit]
Set up unix-like environment in windows
For those with a love-hate relationship with windows...
[edit]
Cygwin
[edit]
Colinux
- Colinux: The Colinux wiki has lots of documentations.
- Installation procedures (details):
- Get the binary
- WinPcap
- Deal with linux image: Need to get the image from from sourceforge and the program Toporesize to change image size.
- Deal with swap file: Here is how you can create swap file.
- Modify configuration file: Copy default.colinux.xml to my.default.colinux.xml and made some modifications. Here is an example. Mine looks like this.
- Start colinux: Crash my computer five times now!!! Here is the windows error report. Crash, crash, crash... Tried many things but still doesn't work. Update the driver. Turn off the video acceleration. Give up for now... Just want to try for the last time, I googled "colinux stop error" and found a thread talking about this. The culprit is "DEP" (Data Execeution Prevention). Need to modify boot.ini to set "/noexecute=alwaysoff" so it is off. This probably can also be done through control panel-system-advanced-DEP and make some exceptions. But I am not sure which one it should be... Anyway, it excutes now - Shiu 17:14, 25 July 2006 (EDT).
[edit]
Access your home directory in calculon
- Under construction...
[edit]
Computing job submission
We have access to two computing cluster resources. The first is the MSU High Performance Computing Center. The second is the in-house cluster Calculon. More info on how to use them is detailed below. Both requires SSH shell. If you don't have ssh program, try SSH Secure Shell or PUTTY.
[edit]
HPCC
HPCC has a well-developed site with extensive documentation. Below is the basics:
tutoring of HPCC Workshop_06March06.pdf
- 1. Login
- ssh -X hpc.msu.edu -l username (username is probably your MSU ID).
- 2. Choose architecture
- Symmetric Multiple Processor system - HPCC has an SMP system called green with 64 processors and 512GB RAM. Once login to HPC in step 1, you can ssh to white which provide qsub interface to green. You can't directly ssh to green.
- Cluster system - HPCC also house a 128 node cluster each with 4 cores and 8GB RAM. The cluster system can be accessed using qsub through either shaw or wilson, both can be ssh'ed directly once login to HPC.
- 3. Run jobs
- HPC use qsub but this one is a bit different from the one calculon use. Particularly, there is no -cwd flag so in the shell script paths are needed for every single file or executables. I am sure there is way around this but don't have time to find out. Let me know if you find out a way so execution can be done in the working directory.
qsub -d path --Zou 16:12, 15 May 2008 (EDT)
[edit]
For green
How to submit a job. There are several queues.
Name_of_queue max#cpu/a_user max#cpu/a_job max_walltime/a_job grean_large 16 16 6 hours green_small 16 8 24 hours green_mtime 4 4 48 hours green_long 4 1 168 hours
Depending on the contents of job, the queue will be changed. If you have shell file "exe_run", you can submit the file as follows.
qsub -l ncpus=16,walltime=6:00:00 ./exe_run
This job will work in "green_large".
[edit]
Calculon
The address of our cluster is: calculon.plantbiology.msu.edu. Use your user name and password to login to the cluster main node by ssh:
ssh username@calculon.plantbiology.msu.edu -p <ask>
Once you login, There are three ways you can run jobs on the cluster:
- 1. Run on the main node, calculon
- This is ok only if the job is short and doesn't require substantial resources. Like testing your code or some unix commands or shell scripts.
- 2. Login to the compute node and run things
- This is ok if you have one resource intensive job. If you have multiple jobs, you need to use the third option. From calculon you can login to compute-0-0, compute-0-1, and compute-0-2 run things. Compute-0-3 is a high memory node that is reserved for running memory intensive applications, such as microarary data proecssing or if your run needs more than 2GB of memory.
- 3. Batch submission
- Our cluster uses Sun GridEngine for job management. The basic usage of GridEngine can be found here.
restart calculon cluster scroll back click twice if one of the compute does not response, restart it. 1.switch on the compute 2.open the right door 3.press the button, until it is shut off 4.press the button again to start.
Below is a very much simplified but working procedure:
- Generate a shell script: It is a file with at least 2 lines, the second line is the command you wish to run -
#!/bin/sh ls
- Submit job: Using the command qsub, you can submit a job using the shell script you just generated -
qsub -cwd your_script
Here -cwd flag is for current working directory. This ensures the outputs are saved in the directory you call qsub.
- Check job progress: Use one of the following -
qstat qstat -F
- Completion: One the job is done, you can check if there is any error and output in two files.
- Error file - script_name.eXXXX. Here XXXX is the job id number, this file reports errors during execution.
- Output file - script_name.oXXXX. This file contains the stdout from the program you called.
less script_name.eXXXX less script_name.oXXXX
[edit]
Common software among workstations
- Firefox
- Thunderbird
- AVG antivirus
- Zonealarm
- Acrobat - one workstation
- Illustrator - one workstation
- Endnote
