htsworkflow.git
8 years agoUse a re.split so we can accept both ', ' and ',' between list items.
Diane Trout [Wed, 23 Sep 2015 21:57:32 +0000 (14:57 -0700)]
Use a re.split so we can accept both ', ' and ',' between list items.

8 years agoindicate we want text not bytes
Diane Trout [Tue, 8 Sep 2015 23:38:12 +0000 (16:38 -0700)]
indicate we want text not bytes

8 years agocovert to using python3 config.get fallback=
Diane Trout [Tue, 8 Sep 2015 23:37:48 +0000 (16:37 -0700)]
covert to using python3 config.get fallback=

8 years agoconvert from optparse to argparse
Diane Trout [Tue, 8 Sep 2015 23:37:03 +0000 (16:37 -0700)]
convert from optparse to argparse

8 years agoadd some documentation -- shocking I know
Diane Trout [Tue, 8 Sep 2015 23:36:25 +0000 (16:36 -0700)]
add some documentation -- shocking I know

8 years agopython3 byte vs str compatibility for constructing document data urls
Diane Trout [Mon, 20 Jul 2015 18:37:23 +0000 (11:37 -0700)]
python3 byte vs str compatibility for constructing document data urls

8 years agoscript to help remigrate from old sqlite database to new server
Diane Trout [Mon, 20 Jul 2015 18:35:58 +0000 (11:35 -0700)]
script to help remigrate from old sqlite database to new server

8 years agoconfiguration file largely only useful for emacs elpy users
Diane Trout [Mon, 20 Jul 2015 18:35:18 +0000 (11:35 -0700)]
configuration file largely only useful for emacs elpy users

8 years agosort load_accession to make it easier to find things
Diane Trout [Mon, 20 Jul 2015 18:34:25 +0000 (11:34 -0700)]
sort load_accession to make it easier to find things

8 years agoremove unused imports
Diane Trout [Thu, 28 May 2015 18:27:38 +0000 (11:27 -0700)]
remove unused imports

8 years agorename datarun to sequencing run, and propogate the change all over the place
Diane Trout [Tue, 26 May 2015 23:58:49 +0000 (16:58 -0700)]
rename datarun to sequencing run, and propogate the change all over the place

8 years agoset a created datetime in the factory, since I removed the default from the model
Diane Trout [Tue, 26 May 2015 23:57:54 +0000 (16:57 -0700)]
set a created datetime in the factory, since I removed the default from the model

8 years agoremove unused import
Diane Trout [Thu, 21 May 2015 23:13:27 +0000 (16:13 -0700)]
remove unused import

8 years agoload in our encode3 accession numbers seperately from the migration system
Diane Trout [Thu, 21 May 2015 23:13:15 +0000 (16:13 -0700)]
load in our encode3 accession numbers seperately from the migration system

8 years agoadd a few common accession agencies to the standard dataset
Diane Trout [Thu, 21 May 2015 23:12:46 +0000 (16:12 -0700)]
add a few common accession agencies to the standard dataset

8 years agoAdd AccessionAgency and LibraryAccession
Diane Trout [Thu, 21 May 2015 23:11:29 +0000 (16:11 -0700)]
Add AccessionAgency and LibraryAccession

For all your library accessioning needs.

8 years agolocalize all the RDF imports and use skipUnless to avoid tests when we don't have...
Diane Trout [Wed, 20 May 2015 21:56:47 +0000 (14:56 -0700)]
localize all the RDF imports and use skipUnless to avoid tests when we don't have redland installed

8 years agoremove commented out code
Diane Trout [Wed, 20 May 2015 21:32:19 +0000 (14:32 -0700)]
remove commented out code

8 years agoimport specific names instead of import factory
Diane Trout [Wed, 20 May 2015 21:31:22 +0000 (14:31 -0700)]
import specific names instead of import factory

8 years agoremove some unused imports
Diane Trout [Wed, 20 May 2015 21:28:55 +0000 (14:28 -0700)]
remove some unused imports

8 years agoremove commented out code
Diane Trout [Wed, 20 May 2015 21:24:03 +0000 (14:24 -0700)]
remove commented out code

8 years agoremove unused imports
Diane Trout [Tue, 12 May 2015 23:56:21 +0000 (16:56 -0700)]
remove unused imports

8 years agoflake8 whitespace cleanup
Diane Trout [Tue, 12 May 2015 23:55:32 +0000 (16:55 -0700)]
flake8 whitespace cleanup

8 years agoremove unused imports
Diane Trout [Tue, 12 May 2015 23:45:20 +0000 (16:45 -0700)]
remove unused imports

8 years agoflake8 whitespace cleanup
Diane Trout [Tue, 12 May 2015 23:40:58 +0000 (16:40 -0700)]
flake8 whitespace cleanup

8 years agoflake8 whitespace cleanup
Diane Trout [Tue, 12 May 2015 21:35:42 +0000 (14:35 -0700)]
flake8 whitespace cleanup

8 years agomove comments out of experiments_flowcell flowcell id and into lane status
Diane Trout [Tue, 12 May 2015 19:18:47 +0000 (12:18 -0700)]
move comments out of experiments_flowcell flowcell id and into lane status

8 years agoremove unused import
Diane Trout [Tue, 12 May 2015 19:17:03 +0000 (12:17 -0700)]
remove unused import

8 years agoflake8 whitespace cleanup
Diane Trout [Tue, 12 May 2015 19:16:31 +0000 (12:16 -0700)]
flake8 whitespace cleanup

8 years agoflake8 whitespace change
Diane Trout [Tue, 12 May 2015 18:52:00 +0000 (11:52 -0700)]
flake8 whitespace change

8 years agorecreate samples initial migration as strings instead of bytestrings
Diane Trout [Tue, 12 May 2015 18:40:53 +0000 (11:40 -0700)]
recreate samples initial migration as strings instead of bytestrings

8 years agorecreate initial migration as strings instead of bytestrings
Diane Trout [Tue, 12 May 2015 18:39:47 +0000 (11:39 -0700)]
recreate initial migration as strings instead of bytestrings

8 years agomove experiment templates into the experiments app
Diane Trout [Tue, 12 May 2015 18:38:12 +0000 (11:38 -0700)]
move experiment templates into the experiments app

9 years agochange the default for the DJANGO_SETTINGS_MODULE to be a little more flexible
Diane Trout [Tue, 5 May 2015 23:31:13 +0000 (16:31 -0700)]
change the default for the DJANGO_SETTINGS_MODULE to be a little more flexible

9 years agosimplify library report
Diane Trout [Tue, 5 May 2015 23:29:40 +0000 (16:29 -0700)]
simplify library report

9 years agoupdate docstring
Diane Trout [Tue, 5 May 2015 23:25:07 +0000 (16:25 -0700)]
update docstring

9 years agoadd in missing import
Diane Trout [Tue, 5 May 2015 23:24:55 +0000 (16:24 -0700)]
add in missing import

9 years agoadd in some missing imports
Diane Trout [Tue, 5 May 2015 23:24:25 +0000 (16:24 -0700)]
add in some missing imports

9 years agoRemove unused imports
Diane Trout [Tue, 5 May 2015 22:02:55 +0000 (15:02 -0700)]
Remove unused imports

9 years agoadjust whitespace
Diane Trout [Tue, 5 May 2015 21:41:10 +0000 (14:41 -0700)]
adjust whitespace

9 years agoAdd more tests for lanes_for
Diane Trout [Tue, 5 May 2015 21:21:24 +0000 (14:21 -0700)]
Add more tests for lanes_for

9 years agoMove lanes_for from samples to experiments
Diane Trout [Tue, 5 May 2015 21:18:41 +0000 (14:18 -0700)]
Move lanes_for from samples to experiments

9 years agoUpdate dependency list
Diane Trout [Tue, 5 May 2015 20:38:35 +0000 (13:38 -0700)]
Update dependency list

Also adjust the formatting a bit.

9 years agomake version utf/python3 friendly
Diane Trout [Tue, 5 May 2015 20:37:14 +0000 (13:37 -0700)]
make version utf/python3 friendly

9 years agoremove extra blank line
Diane Trout [Tue, 5 May 2015 20:36:04 +0000 (13:36 -0700)]
remove extra blank line

9 years agoOops I had a test function name collision. Rename one so it gets executed
Diane Trout [Mon, 6 Apr 2015 22:21:40 +0000 (15:21 -0700)]
Oops I had a test function name collision. Rename one so it gets executed

9 years agoDrop support for simplejson
Diane Trout [Mon, 6 Apr 2015 22:07:44 +0000 (15:07 -0700)]
Drop support for simplejson

We're only targeting python2.7+ & python3.3+ so I can drop
that archaic code.

(Also gets me some more test coverage)

9 years agoRemove suprious comma. Fixing a tuple <= int bug
Diane Trout [Mon, 6 Apr 2015 22:03:14 +0000 (15:03 -0700)]
Remove suprious comma. Fixing a tuple <= int bug

python 2 didn't catch this type error. Thankfully python 3 does.

9 years agoMissed a couple of smart_text wrappings of django test client response.content.
Diane Trout [Mon, 6 Apr 2015 21:53:19 +0000 (14:53 -0700)]
Missed a couple of smart_text wrappings of django test client response.content.

(it comes back as bytes in python3, which causes trouble for librdf)

9 years agoUpdate test to be more robust to changing dictionary order.
Diane Trout [Mon, 6 Apr 2015 21:52:27 +0000 (14:52 -0700)]
Update test to be more robust to changing dictionary order.

The test used to be stable in python2 days, but lets just
use sets to make sure it wont fail in the future.

9 years agofix a white space issue
Diane Trout [Mon, 6 Apr 2015 21:52:12 +0000 (14:52 -0700)]
fix a white space issue

9 years agoconvert non-unit-tested module to use newer raise Exception(msg) syntax
Diane Trout [Mon, 6 Apr 2015 21:51:46 +0000 (14:51 -0700)]
convert non-unit-tested module to use newer raise Exception(msg) syntax

9 years agoconvert several not covered by unit-test modules to use print function
Diane Trout [Mon, 6 Apr 2015 21:50:03 +0000 (14:50 -0700)]
convert several not covered by unit-test modules to use print function

9 years agoBe more careful looking up keys when the fastqname is uninitialized.
Diane Trout [Mon, 6 Apr 2015 20:50:36 +0000 (13:50 -0700)]
Be more careful looking up keys when the fastqname is uninitialized.

By removing the default return of None from getitem, the is_valid test
failed when the object was uninitialized.

9 years agoadd post_spreadsheet to create encoded objects from a excel like spreadsheet
Diane Trout [Tue, 31 Mar 2015 22:38:47 +0000 (15:38 -0700)]
add post_spreadsheet to create encoded objects from a excel like spreadsheet

9 years agoalso validate documents before creating them
Diane Trout [Tue, 31 Mar 2015 22:38:19 +0000 (15:38 -0700)]
also validate documents before creating them

9 years agoactually complain if we get the wrong type
Diane Trout [Tue, 31 Mar 2015 22:37:57 +0000 (15:37 -0700)]
actually complain if we get the wrong type

9 years agoadd a docstring to validate
Diane Trout [Tue, 31 Mar 2015 22:37:11 +0000 (15:37 -0700)]
add a docstring to validate

9 years agoAdd get_accession_name to use collection name to find commonly used spreadsheet acces...
Diane Trout [Tue, 31 Mar 2015 22:35:52 +0000 (15:35 -0700)]
Add get_accession_name to use collection name to find commonly used spreadsheet accesssion column name

9 years agosupport trying to figure out object type based on what collection its going to be...
Diane Trout [Tue, 31 Mar 2015 22:35:19 +0000 (15:35 -0700)]
support trying to figure out object type based on what collection its going to be posted to

9 years agoMerge branch 'django1.7' of mus.cacr.caltech.edu:htsworkflow into django1.7
Diane Trout [Fri, 27 Mar 2015 18:13:50 +0000 (11:13 -0700)]
Merge branch 'django1.7' of mus.cacr.caltech.edu:htsworkflow into django1.7

9 years agoSwitch encode3 submitter to use aws.
Diane Trout [Fri, 27 Mar 2015 18:11:11 +0000 (11:11 -0700)]
Switch encode3 submitter to use aws.

I needed to write a submission class to support submitting to
aws, and of course there's a slightly different set of metadata
they want to know for submission.

9 years agoImport flowcell details into the triple store from a lane
Diane Trout [Fri, 27 Mar 2015 18:07:06 +0000 (11:07 -0700)]
Import flowcell details into the triple store from a lane

9 years agostore flowcell name in the triple store if its known
Diane Trout [Fri, 27 Mar 2015 18:05:05 +0000 (11:05 -0700)]
store flowcell name in the triple store if its known

9 years agorecord file size information for a file in the triple store
Diane Trout [Fri, 27 Mar 2015 18:03:45 +0000 (11:03 -0700)]
record file size information for a file in the triple store

9 years agoAdjust logging overrides and move django.setup to after the override.
Diane Trout [Fri, 27 Mar 2015 18:01:15 +0000 (11:01 -0700)]
Adjust logging overrides and move django.setup to after the override.

(Otherwise the extra logging doesn't do much.

9 years agogetitem should only return the key
Diane Trout [Fri, 27 Mar 2015 17:58:35 +0000 (10:58 -0700)]
getitem should only return the key

9 years agoclean up a bit of whitespace
Diane Trout [Fri, 27 Mar 2015 17:53:07 +0000 (10:53 -0700)]
clean up a bit of whitespace

9 years agoconfigure using a local postgres install
Diane Trout [Fri, 20 Mar 2015 23:33:26 +0000 (16:33 -0700)]
configure using a local postgres install

9 years agoConvert reads to an integer, if if it was a float in the xml.
Diane Trout [Fri, 20 Mar 2015 23:31:53 +0000 (16:31 -0700)]
Convert reads to an integer, if if it was a float in the xml.

Some versions of the pipeline include a trailing .0 for read length,
this made python 3s parser unhappy. so convert to a float and then
truncate to work around the type issue.

9 years agoActually comare an integer (length of list) to an integer 0.
Diane Trout [Fri, 20 Mar 2015 23:30:14 +0000 (16:30 -0700)]
Actually comare an integer (length of list) to an integer 0.

Why was list() > 0 ever valid code?

9 years agoThe empty list is now not found
Diane Trout [Fri, 20 Mar 2015 23:29:24 +0000 (16:29 -0700)]
The empty list is now not found

No I don't know why.

9 years agoDrop library id encoding
Diane Trout [Fri, 20 Mar 2015 23:27:19 +0000 (16:27 -0700)]
Drop library id encoding

9 years agoUse regex to make sure scripts end with the right command
Diane Trout [Fri, 20 Mar 2015 23:26:04 +0000 (16:26 -0700)]
Use regex to make sure scripts end with the right command

python 3 doesn't seem to return the path to a .pyc file
like python 2.

9 years agoUse six.moves to find urlencode and urlopen
Diane Trout [Fri, 20 Mar 2015 23:25:20 +0000 (16:25 -0700)]
Use six.moves to find urlencode and urlopen

9 years agofilter out None c.times in list of potential times
Diane Trout [Fri, 20 Mar 2015 23:24:55 +0000 (16:24 -0700)]
filter out None c.times in list of potential times

9 years agouse relative import for submission_test_common
Diane Trout [Fri, 20 Mar 2015 23:23:50 +0000 (16:23 -0700)]
use relative import for submission_test_common

9 years agoTry to detect if an object is a file type on python python 2 & 3.
Diane Trout [Fri, 20 Mar 2015 23:22:06 +0000 (16:22 -0700)]
Try to detect if an object is a file type on python python 2 & 3.

The base type for files changed between the two releases.

9 years agoreplace xrange with enumerate
Diane Trout [Fri, 20 Mar 2015 23:21:35 +0000 (16:21 -0700)]
replace xrange with enumerate

9 years agoUse ''.splitlines() instead of split('\n')
Diane Trout [Fri, 20 Mar 2015 23:19:39 +0000 (16:19 -0700)]
Use ''.splitlines() instead of split('\n')

it seems to handle byte vs text strings a bit better.

9 years agoadapt to python2 returning lists and python3 iterators
Diane Trout [Fri, 20 Mar 2015 23:14:42 +0000 (16:14 -0700)]
adapt to python2 returning lists and python3 iterators

This leads to code like next(iter(obj.values())) instead of
obj.values()[0]. Honestly Im not sure which is less ugly.

Also there's a few places where I have to cast things with list()
to get my tests to pass.

9 years agouse sorted on the iterator instead of an in-place sort
Diane Trout [Fri, 20 Mar 2015 21:46:27 +0000 (14:46 -0700)]
use sorted on the iterator instead of an in-place sort

9 years agoUse max on the iterator to find the last key
Diane Trout [Fri, 20 Mar 2015 21:45:29 +0000 (14:45 -0700)]
Use max on the iterator to find the last key

9 years agoadd more from __future__ import print_function, unicode_literals
Diane Trout [Fri, 20 Mar 2015 21:24:41 +0000 (14:24 -0700)]
add more from __future__ import print_function, unicode_literals

9 years agoCall print(msg) instead of print msg
Diane Trout [Fri, 20 Mar 2015 21:20:27 +0000 (14:20 -0700)]
Call print(msg) instead of print msg

9 years agospecial case unicode handling that differs between python2 & 3
Diane Trout [Fri, 20 Mar 2015 21:19:05 +0000 (14:19 -0700)]
special case unicode handling that differs between python2 & 3

9 years agoReplace some unicode() with str()
Diane Trout [Fri, 20 Mar 2015 21:18:33 +0000 (14:18 -0700)]
Replace some unicode() with str()

9 years agoUse isinstance(object, (types)) pattern instead of type(object) == types.Type
Diane Trout [Fri, 20 Mar 2015 21:13:54 +0000 (14:13 -0700)]
Use isinstance(object, (types)) pattern instead of type(object) == types.Type

9 years agoSpecify text vs binary mode for opening files.
Diane Trout [Fri, 20 Mar 2015 20:47:36 +0000 (13:47 -0700)]
Specify text vs binary mode for opening files.

(And another smart_text cast sliped in)

9 years agoprefer native string types.
Diane Trout [Fri, 20 Mar 2015 20:42:12 +0000 (13:42 -0700)]
prefer native string types.

Things get really messy when trying to pass unicode strings
to redland in python2. I found it simpler to just try and keep
using the native string type.

So this patch is largely replacing unicode() to str() and
changing u'' to ''.

9 years agoupdate more raise exceptions to Exception(message) syntax
Diane Trout [Fri, 20 Mar 2015 20:30:36 +0000 (13:30 -0700)]
update more raise exceptions to Exception(message) syntax

9 years agoUse django.utils.encode smart_text or smart_str for intelligent unicode handling
Diane Trout [Thu, 19 Mar 2015 21:48:46 +0000 (14:48 -0700)]
Use django.utils.encode smart_text or smart_str for intelligent unicode handling

some things like writing files or talking to redland are
quite picky about whether or not they're unicode or ascii strings.

9 years agoimport urljoin from six.moves.urllib.parse
Diane Trout [Wed, 18 Mar 2015 23:52:28 +0000 (16:52 -0700)]
import urljoin from six.moves.urllib.parse

9 years agouse JsonResponse instead of making a json response by hand using HttpResponse
Diane Trout [Wed, 18 Mar 2015 23:50:34 +0000 (16:50 -0700)]
use JsonResponse instead of making a json response by hand using HttpResponse

9 years agodrop support for simplejson. were only caring about python 2.7+ and 3.3+
Diane Trout [Wed, 18 Mar 2015 23:49:31 +0000 (16:49 -0700)]
drop support for simplejson. were only caring about python 2.7+ and 3.3+

9 years agoimport ConfigParser classes from six.moves.configparser
Diane Trout [Wed, 18 Mar 2015 23:46:47 +0000 (16:46 -0700)]
import ConfigParser classes from six.moves.configparser

9 years agouse absolute_import
Diane Trout [Wed, 18 Mar 2015 23:45:08 +0000 (16:45 -0700)]
use absolute_import

9 years agonew json code returns [] when there are no lanes
Diane Trout [Tue, 3 Feb 2015 00:30:31 +0000 (16:30 -0800)]
new json code returns [] when there are no lanes

9 years agoconvert to unicode_literals
Diane Trout [Tue, 3 Feb 2015 00:24:18 +0000 (16:24 -0800)]
convert to unicode_literals

convert all the string handling to unicode.
to make the RDF stuff work I needed to mark those strings
as byte arrays

9 years agotake advantage of absolute_import to simplify import statements
Diane Trout [Mon, 2 Feb 2015 22:54:28 +0000 (14:54 -0800)]
take advantage of absolute_import to simplify import statements