Diane Trout [Wed, 15 Jan 2014 01:01:45 +0000 (17:01 -0800)]
remove some trailing whitespace
Diane Trout [Wed, 15 Jan 2014 01:01:18 +0000 (17:01 -0800)]
remove some trailing whitespace
Diane Trout [Wed, 15 Jan 2014 00:58:50 +0000 (16:58 -0800)]
Add option to get_model to disable RDF graph context.
At least with the version of librdf I was using the context
graph really slowed adding tripes down.
Diane Trout [Wed, 8 Jan 2014 18:04:04 +0000 (10:04 -0800)]
Ignore kdevelop & kate project files.
Diane Trout [Wed, 8 Jan 2014 18:03:05 +0000 (10:03 -0800)]
Rename stripNamespace strip_namespace
Diane Trout [Mon, 23 Dec 2013 22:30:36 +0000 (14:30 -0800)]
Merge branch 'master' of mus.cacr.caltech.edu:htsworkflow
Diane Trout [Mon, 23 Dec 2013 22:30:16 +0000 (14:30 -0800)]
update settings for new location of WSGI_APPLICATION, and add a reminder about where to set debug
Diane Trout [Mon, 23 Dec 2013 22:29:26 +0000 (14:29 -0800)]
fix quote typos with the {% static %} template tag
Diane Trout [Mon, 23 Dec 2013 22:28:30 +0000 (14:28 -0800)]
Move wsgi into the actual project root.
Diane Trout [Tue, 17 Dec 2013 19:50:12 +0000 (11:50 -0800)]
remove some test code debugging print statements
Diane Trout [Tue, 17 Dec 2013 19:44:26 +0000 (11:44 -0800)]
Update WSGI_APPLICATION to point to the updated location of the wsgi script
Diane Trout [Tue, 17 Dec 2013 19:32:20 +0000 (11:32 -0800)]
Inventory works, even though its was hidden between two non-working modules
Diane Trout [Tue, 17 Dec 2013 19:29:25 +0000 (11:29 -0800)]
move an example result directory into git and use it for testing
Diane Trout [Tue, 17 Dec 2013 19:20:03 +0000 (11:20 -0800)]
Merge branch 'django1.4'
there was a conflict with my qualifying the load_pipeline_run_xml function call
Conflicts:
htsworkflow/frontend/samples/views.py
Diane Trout [Tue, 17 Dec 2013 19:19:09 +0000 (11:19 -0800)]
Update requirements to require diango 1.6
Diane Trout [Tue, 17 Dec 2013 19:15:09 +0000 (11:15 -0800)]
Start simplyfing settings by using from settings_local import * pattern.
This should be a lot less ugly than my previous implementation
using the ini file. However there are still settings that need to
be migrated over.
Diane Trout [Tue, 17 Dec 2013 19:14:35 +0000 (11:14 -0800)]
add new wsgi.py driver
Diane Trout [Tue, 17 Dec 2013 19:13:24 +0000 (11:13 -0800)]
Update templates to consistently use {% static %} instead of hard coding paths
Also import a few media files that were missing.
Diane Trout [Tue, 17 Dec 2013 19:11:04 +0000 (11:11 -0800)]
Move loading DTD into a function as the call caused a hang when loaded into wsgi
Diane Trout [Tue, 17 Dec 2013 19:10:44 +0000 (11:10 -0800)]
move django imports around, change one function call to a qualified import
Diane Trout [Tue, 17 Dec 2013 19:09:31 +0000 (11:09 -0800)]
Update exception caught for newer django?
Diane Trout [Tue, 17 Dec 2013 19:08:35 +0000 (11:08 -0800)]
use url function call for url definition instead of just a bare tuple
Diane Trout [Tue, 17 Dec 2013 19:05:00 +0000 (11:05 -0800)]
Move use of settings default into the function instead of function definition.
This is because the if its not defined the program wont start up.
Diane Trout [Tue, 10 Dec 2013 23:17:13 +0000 (15:17 -0800)]
Update manage script to django 1.6
Diane Trout [Tue, 10 Dec 2013 23:15:43 +0000 (15:15 -0800)]
Being able to filter on paired end runs might be useful
Diane Trout [Tue, 10 Dec 2013 23:15:20 +0000 (15:15 -0800)]
remove deprecated module django.contrib.databrowse
Diane Trout [Tue, 10 Dec 2013 23:13:51 +0000 (15:13 -0800)]
With 1.6 boolean field changed to default Null, so to be backwards compatible set default false
Diane Trout [Tue, 10 Dec 2013 23:12:42 +0000 (15:12 -0800)]
Django 1.6 is pickier about enforcing referential integrity
Diane Trout [Tue, 10 Dec 2013 23:11:54 +0000 (15:11 -0800)]
Fix urls patterns import to be compatible with django 1.6
Diane Trout [Tue, 10 Dec 2013 00:30:05 +0000 (16:30 -0800)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Tue, 10 Dec 2013 00:13:48 +0000 (16:13 -0800)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Mon, 9 Dec 2013 22:44:10 +0000 (14:44 -0800)]
Add new paired library types to our hackish list for library classifications
Diane Trout [Fri, 6 Dec 2013 00:28:12 +0000 (16:28 -0800)]
Attempt to be robust to not having an alignment in our RunXml file
Diane Trout [Fri, 6 Dec 2013 00:28:12 +0000 (16:28 -0800)]
Attempt to be robust to not having an alignment in our RunXml file
Diane Trout [Fri, 6 Dec 2013 00:02:51 +0000 (16:02 -0800)]
After implementing proper tests get a working implementation of runfolder name from IPAR tree.
My first couple tries didn't work hopefully this one will work
Diane Trout [Fri, 6 Dec 2013 00:02:51 +0000 (16:02 -0800)]
After implementing proper tests get a working implementation of runfolder name from IPAR tree.
My first couple tries didn't work hopefully this one will work
Diane Trout [Thu, 5 Dec 2013 23:28:36 +0000 (15:28 -0800)]
fix typo
Diane Trout [Thu, 5 Dec 2013 23:28:36 +0000 (15:28 -0800)]
fix typo
Diane Trout [Thu, 5 Dec 2013 23:28:36 +0000 (15:28 -0800)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Thu, 5 Dec 2013 23:06:14 +0000 (15:06 -0800)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Thu, 5 Dec 2013 22:45:59 +0000 (14:45 -0800)]
Recent IPAR xml config blocks include the runfolder name
I was previously extracting it in the gerald step, but since we
have a number of runfolders with no aligned directory that wasn't working.
Conflicts:
htsworkflow/pipelines/test/test_runfolder_rta180.py
Diane Trout [Thu, 5 Dec 2013 22:51:06 +0000 (14:51 -0800)]
Update for UCSC's validate manifest 1.7
Diane Trout [Thu, 5 Dec 2013 22:48:46 +0000 (14:48 -0800)]
Provide reporting of what command failed when bigWigInfo isn't available.
Diane Trout [Thu, 5 Dec 2013 22:45:59 +0000 (14:45 -0800)]
Recent IPAR xml config blocks include the runfolder name
I was previously extracting it in the gerald step, but since we
have a number of runfolders with no aligned directory that wasn't working.
Diane Trout [Thu, 5 Dec 2013 22:37:39 +0000 (14:37 -0800)]
Add a way to report what url was failing
Diane Trout [Mon, 2 Dec 2013 19:24:43 +0000 (11:24 -0800)]
Catch and report another class of exception
Diane Trout [Mon, 21 Oct 2013 21:53:56 +0000 (14:53 -0700)]
Merge remote-tracking branch 'brisby/django1.4' into django1.4
Diane Trout [Mon, 21 Oct 2013 21:51:14 +0000 (14:51 -0700)]
Pass additional information to the trackhub.
This allows adding a term to control visibility of tracks.
There's also a hack to pull out the min/max bigWig values when generating
the trackDb.txt file.
It unfortunately depends on having bigWigInfo from UCSC on your path.
Diane Trout [Mon, 21 Oct 2013 20:59:29 +0000 (13:59 -0700)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Thu, 17 Oct 2013 22:03:43 +0000 (15:03 -0700)]
Merge branch 'master' into django1.4
Diane Trout [Thu, 17 Oct 2013 22:02:51 +0000 (15:02 -0700)]
Add command line feature to srf command
this should allow copying out fastq-equivalent files from a run.
Diane Trout [Thu, 17 Oct 2013 21:55:14 +0000 (14:55 -0700)]
Dynamically respond to the number of lanes.
Illumina now has single lane flowcells so the assumtion that there
will be 8 lanes is wrong.
Diane Trout [Thu, 17 Oct 2013 21:31:29 +0000 (14:31 -0700)]
Fix typo.
Diane Trout [Thu, 17 Oct 2013 21:17:52 +0000 (14:17 -0700)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Thu, 17 Oct 2013 21:12:05 +0000 (14:12 -0700)]
Implement a wrapper around bigWigInfo to report bigWig summary info.
(shocking name there I know)
Diane Trout [Fri, 11 Oct 2013 21:58:46 +0000 (14:58 -0700)]
Make it possible to set a label on a track.
This works by setting a term label_template on the filename regex
block. that label_template is then expanded by searching for metadata
in from the related library for that submission.
Diane Trout [Wed, 9 Oct 2013 22:24:18 +0000 (15:24 -0700)]
add replaces / replace_reason generation to trackhub manifest
Diane Trout [Wed, 9 Oct 2013 22:23:41 +0000 (15:23 -0700)]
remove spurious newline
Diane Trout [Tue, 8 Oct 2013 23:50:23 +0000 (16:50 -0700)]
Be more forgiving about the md5sum filename path
Diane Trout [Tue, 8 Oct 2013 23:49:24 +0000 (16:49 -0700)]
Add helpful logging information to explain missing required options.
Diane Trout [Tue, 24 Sep 2013 21:38:39 +0000 (14:38 -0700)]
Share more code to generate result tree.
This was in preparation of trying to support some mechanism
to submit different analysis from one directory somehow.
Diane Trout [Mon, 12 Aug 2013 20:47:04 +0000 (13:47 -0700)]
Update test data for renamed library type.
Diane Trout [Sat, 10 Aug 2013 00:06:16 +0000 (17:06 -0700)]
Add sortOrder and filtering parameters to the trackhub.
The order of trackhub parameters is still fixed,
however it should skip missing options.
Diane Trout [Mon, 22 Jul 2013 20:53:42 +0000 (13:53 -0700)]
Move version finding code into the util directory.
Mostly done because I already have test code in
htsworkflow.util.test
Diane Trout [Mon, 22 Jul 2013 19:53:39 +0000 (12:53 -0700)]
Test changes to submission code.
Also there's some commonality in simulating a submission directory
so refactor that code out to a common module.
Diane Trout [Mon, 22 Jul 2013 19:48:53 +0000 (12:48 -0700)]
Generate manifest.txt files for submitting to ENCODE3.
Change trackhub generation from my previous template version to use
the Daler trackhub code.
This includes a feature to complain if you offer a submission set
name that diesn't exist.
Also the samples query returns all the submission components in
a single query instead of one of a time. (Which is much faster
way of doing things).
Diane Trout [Mon, 22 Jul 2013 19:20:19 +0000 (12:20 -0700)]
turtle writing improvements.
* Update namespaces added to default writer.
* Add a function to generate a default turtle prefix header.
Diane Trout [Mon, 22 Jul 2013 19:16:58 +0000 (12:16 -0700)]
Add a generator that returns analysis nodes from a result map.
It iterates over the submission directories and returns
the fully qualified RDF node for them.
Diane Trout [Sat, 20 Jul 2013 05:50:06 +0000 (22:50 -0700)]
Add python namespace for encode3 RDF namespace
Diane Trout [Wed, 17 Jul 2013 22:28:16 +0000 (15:28 -0700)]
Add function to list the names for submissions from the RDF model.
Currently the model is ill-specified and the name entries
are just pointing at the list of per-library directory names.
Make sure the end of the submission name doesn't have URL seperator
characters.
Diane Trout [Wed, 17 Jul 2013 00:02:30 +0000 (17:02 -0700)]
Add function to parse scp / ssh style URLs.
Also move some tests around from htsworkflow.util.url
Diane Trout [Tue, 16 Jul 2013 22:04:52 +0000 (15:04 -0700)]
Deleting commented out code.
Came up with a better library filter strategy
Diane Trout [Mon, 8 Jul 2013 22:02:42 +0000 (15:02 -0700)]
Update test to work with species -> species_name rename.
Diane Trout [Wed, 3 Jul 2013 17:57:10 +0000 (10:57 -0700)]
Generate manifest files for ENCODE3
I added a new option to the trackhub generation script.
There were some changes to the model generation to capture
relative path names and add the library URI to files to
make some queries faster.
Diane Trout [Mon, 1 Jul 2013 23:02:46 +0000 (16:02 -0700)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
Diane Trout [Mon, 1 Jul 2013 22:59:10 +0000 (15:59 -0700)]
Add the option to copy tree in addition to making a symlink tree from elsewhere.
Also don't copy subdirectories in an analysis directory tree being copied
from elsewhere.
Diane Trout [Mon, 1 Jul 2013 22:57:12 +0000 (15:57 -0700)]
Initial attempt to start generating trackHubs and manifest files.
Diane Trout [Mon, 1 Jul 2013 22:53:53 +0000 (15:53 -0700)]
type checking more detailed than Literal doesn't work well
Diane Trout [Fri, 28 Jun 2013 18:36:31 +0000 (11:36 -0700)]
Add option to copy source files for a submission.
Sometimes it may be worth while to keep a copy of the
files being submitted.
Diane Trout [Tue, 18 Jun 2013 18:59:13 +0000 (11:59 -0700)]
Further improve reliability of make_tree_from.
The previous update assumed that it was going to be running
in the target directory. which wasn't true in the test cases.
I also updated the test case to handle both a base filename and
an absolute pathname for the result map.
Diane Trout [Tue, 18 Jun 2013 00:19:22 +0000 (17:19 -0700)]
copy_tree_from wasn't actually making any symlinks.
It turns out at some point the results class switched to using
full paths. this meant my os.path.join's didn't work as it
didn't modify fullly qualified paths.
This patch converts the result lib path to a relative path based
on the destination so it can compute source paths more easily.
Diane Trout [Tue, 18 Jun 2013 00:18:00 +0000 (17:18 -0700)]
Update tests for new version of redland rdf lib.
Also change from failUnless to assert
Diane Trout [Tue, 18 Jun 2013 00:09:29 +0000 (17:09 -0700)]
Remove deprecated adminmedia template tag from loader.
Since these templates didn't actually use the feature
I didn't bother replacing adminmedia with staticfiles
Diane Trout [Tue, 18 Jun 2013 00:06:13 +0000 (17:06 -0700)]
Replace deprecated django.contrib adminmedia with staticfiles.
Also update url template tag for django 1.5 syntax.
Diane Trout [Tue, 18 Jun 2013 00:00:40 +0000 (17:00 -0700)]
Use proper User model import.
The previous location was an unsupported alias.
Diane Trout [Fri, 1 Feb 2013 01:04:58 +0000 (17:04 -0800)]
Preliminary implementation of trackDb generation.
This is super preliminary importand report parts are
hard coded instead of being detected properly.
Diane Trout [Fri, 1 Feb 2013 01:03:58 +0000 (17:03 -0800)]
Forgot to import an exception I used
Diane Trout [Fri, 1 Feb 2013 01:01:02 +0000 (17:01 -0800)]
Make a sample key list to go along with our lane list
At some point I had to add the sample key which could link
lane, library and index together. Some code know expects that
class so I needed to create a "standard" list and pass it in.
diane [Wed, 16 Jan 2013 01:29:06 +0000 (17:29 -0800)]
Add more entries to genome version detection map.
Renamed hg19 to hg18.
Added saccer1, Anopheles_gambiae detection
Diane Trout [Wed, 16 Jan 2013 01:00:22 +0000 (17:00 -0800)]
Support scanning HiSeq runs with multiple analyses.
It extends the previous C1-100 directory name with the concept of
a suffix extension to that name. The suffix is gathered from whatever the
user came up with their own Aligned/Unaligned directory names.
I discovered I'd previously been calling the
"run_{flowcell id}_{timestamp}.xml" filename the runfolder name. Which is
dumb, as that's a filename, not a name. So this patch renamed it.
(Since I needed to clean up some of the names to implement the above
"run_dirname" functionality.
since I was testing the run.name in the test_runfolders* I needed
to fix those. (And I'm really regretting my cut-n-paste programming).
Diane Trout [Wed, 16 Jan 2013 00:55:20 +0000 (16:55 -0800)]
Thrown an exception if we can't parse base calling directory.
While working on making the alignment parsing optional. I made a mistake
copying some files around, that led to an unparseable base call directory.
I thought I should try to catch and report that error condition.
Diane Trout [Tue, 15 Jan 2013 23:09:56 +0000 (15:09 -0800)]
Start making documentation for htsworkflow.
This is a tiny start toward creating documentation for
htsworkflow, mostly I was getting confused about some of my
classes and thought I should try to get some API documentation
going.
Since I wanted to make sure I had the docstring syntax right I needed
some way to actually build the documentation, and so I might as well
commit what little documentation I created.
Diane Trout [Wed, 9 Jan 2013 00:11:13 +0000 (16:11 -0800)]
A better resolution to a possible circular dependency.
The runfolder subdirectory processing tools (firecrest, bustard, gerald, etc.)
were importing runfolder for some common constants, however runfolder
imported them to actually build the runfolder structure.
My previous solution was to only include the imports for the
sub-directory processing in the function that used them.
However that lead to needing nested functions which seemed confusing.
What I did was move the common constants into pipelines.__init__
and just imported them from there.
Diane Trout [Wed, 9 Jan 2013 00:03:34 +0000 (16:03 -0800)]
Don't accidentally transform an object into a tuple.
I had an extra ',' that was turning a simple assignment
into assignment of a tuple containing the variable I was expecting.
needless to say this cause trouble.
Diane Trout [Tue, 8 Jan 2013 01:51:28 +0000 (17:51 -0800)]
Merge branch 'django1.4' of mus.cacr.caltech.edu:htsworkflow into django1.4
I fixed one of the time-stamp formats on both development machines.
Conflicts:
htsworkflow/pipelines/gerald.py
Diane Trout [Tue, 8 Jan 2013 01:46:40 +0000 (17:46 -0800)]
Try to make Aligned result directories optional in hiseq runs.
The previous implementation tried to match Aligned & Unaligned
directories by parsing the Aligned directories config file for
its the unaligned raw sequence directory.
Needless to say that didn't work if there wasn't an Aligned
directory.
This version tries to match them by comparing the suffix in
Aligned<Suffix> and Unaligned<Suffix>. Then the runfolder generation
code will still generate a runfolder if there's no aligned directory.
Diane Trout [Tue, 8 Jan 2013 01:46:06 +0000 (17:46 -0800)]
Add unaligned stats files to hiseq test case
Diane Trout [Tue, 8 Jan 2013 01:30:54 +0000 (17:30 -0800)]
Tweak timestamp format.
Diane Trout [Fri, 14 Dec 2012 01:16:20 +0000 (17:16 -0800)]
Gerald's time-stamp format was inconsistent.
The different os / python versions had different defaults for
'%c'. I'd previously changed the read function, but not the
generation function. Also it didn't look quite like some
of my timestamps in my files.
So now both creating the time stamp and parsing the time
stamp are using the same date string.
Diane Trout [Fri, 14 Dec 2012 00:58:56 +0000 (16:58 -0800)]
Make my ChangeList sub-class compatibile with Django 1.3
Django 1.3's django.contrib.admin.view.main.ChangeList
class takes one fewer parameter than the 1.4 version, as
does the get_query_set function.
I solved this by testing the django.VERSION and adding
the extra paramemeter to a dictionary and calling with
**kwarg expansion.
Yes it is dirty.