| Time |
S |
Nick |
Message |
| 00:08 |
|
|
skay joined #dvn |
| 00:50 |
|
|
JonathanNeal joined #dvn |
| 03:51 |
|
|
axfelix joined #dvn |
| 11:28 |
|
|
ruebot joined #dvn |
| 14:33 |
|
|
LyndsySimon joined #dvn |
| 15:53 |
|
skay |
pdurbin: I am hoping everyone who is interested in the data versioning and other data discussions all get together on a mailing list so that everyone can share ideas |
| 15:57 |
|
pdurbin |
skay: new comments! just added! read them, read them! https://docs.google.com/document/d/18WDIS8hrFJvMJBcnRuQ8NfD-VxGq32vJ9WwlEgyyWZs/edit?usp=sharing |
| 16:04 |
|
skay |
pdurbin: I saw! I set hte doc so that it would send me email anytime someone leaves a comment. I hadn't found that setting before. I like google docs more now. |
| 16:04 |
|
pdurbin |
skay: ok. but where are YOUR comments? |
| 16:06 |
|
skay |
:p |
| 16:07 |
|
skay |
I've got the problem where I only want to leave amazingly profound comments |
| 16:13 |
|
pdurbin |
uh huh |
| 16:13 |
|
pdurbin |
that's why IRC is nice |
| 16:13 |
|
pdurbin |
:) |
| 16:17 |
|
skay |
yeah :) |
| 16:18 |
|
skay |
I am thinking I disagree with you about not storing metadata in a metadata kind of file in the repo, but I need to read your doc in more depth to make sure I got the gist of what you meant |
| 16:19 |
|
skay |
I like the niceness of being able to see diffs in text files with info about a line that changes... so for example, saving a pretty printed json file where you could see one field changing |
| 16:19 |
|
skay |
but I probably just skimmed and forgot what you said and what I am saying isn't applicable |
| 16:19 |
|
pdurbin |
skay: but should we let users edit that JSON and then commit and push? |
| 16:20 |
|
pdurbin |
(in theory) |
| 16:20 |
|
pdurbin |
(thought experiment!) |
| 16:21 |
|
skay |
pdurbin: no, I would want to restrict it to being touched via an api that builds it |
| 16:21 |
|
skay |
but if you are using a repo as the store, storing it in textual format where I get to take advantage of diffs is really nice |
| 16:22 |
|
skay |
it's like the idea of those config files that get generated and then say, hey! you! don't edit this! it was generated! |
| 16:22 |
|
skay |
but you still get to access the file and look it like you get to do normal files on a file system |
| 16:22 |
|
pdurbin |
skay: if it's in the git repo you can edit it and commit and push. that's the point. should it (metadata) be in the repo? as JSON or whatever |
| 16:22 |
|
skay |
I think you should restrict actions to the repo, and if that feature does not exist, then make it exist |
| 16:22 |
|
pdurbin |
it's not me setting these rules. it's just how git works |
| 16:23 |
|
skay |
you would have a layer on top of git that does the commit stuff |
| 16:23 |
|
pdurbin |
the commit stuff |
| 16:23 |
|
pdurbin |
skay: prototype it for me :) |
| 16:23 |
|
skay |
you would not allow just any user to interact with git unless you want to allow them to shoot off their feed with a machine gun |
| 16:24 |
|
|
LyndsySimon joined #dvn |
| 16:24 |
|
skay |
take sumatra for example, people track simulation runs that generate data, sumatra tracks the diffs in the way that sumatra defines diffs. you can export a json file of those records |
| 16:24 |
|
skay |
one moment while I find the source code for you |
| 16:25 |
|
skay |
https://bitbucket.org/apdavison/sumatra/src/404ec764a305b5346de2e02569cf167cb59cf762/sumatra/recordstore/serialization.py?at=default#cl-14 |
| 16:26 |
|
skay |
now let's consider the case where there is something you are backing by git or some other thing that can store things in a textual way, and now let's consider the case where you have json thing to store, such as what might be generated by a sumatra user |
| 16:27 |
|
skay |
it will be the case that you will have a UI that users get to have access to, but your UI will sit on top of a service that does not give the user full access to the service being called |
| 16:27 |
|
skay |
sumatra or something else in that role generates some metadata that is json |
| 16:27 |
|
skay |
and stores it in version control |
| 16:27 |
|
skay |
your service layer does the transformation to let the user see it |
| 16:28 |
|
skay |
because your service layer even delivers plain old diffs to the user |
| 16:28 |
|
skay |
the user does not have any write privs, but perhaps if you are feeling nice, you can give them some read privs to the backend store |
| 16:28 |
|
skay |
here user, have a read only thingee that is the backend store of your stuff |
| 16:28 |
|
skay |
GET the stuff! |
| 16:29 |
|
skay |
PUT the stuff? POST the stuff? yeah right I don't think so. |
| 16:29 |
|
skay |
or git protocol or whatever protocol you want |
| 16:29 |
|
skay |
please not jini/pojo |
| 16:30 |
|
skay |
I wouldn't actually take sumatra records and commit changes over them |
| 16:31 |
|
skay |
I was just usng one of those as an example of some metadata about some use of data |
| 16:31 |
|
skay |
I was thinking of taking hte sumatra project and making something like a service interface to it |
| 16:32 |
|
skay |
also, irc is not completely transient when you log the channel so this means I say nonsense and it is on record |
| 16:32 |
|
skay |
later when I know more I will be so embarassed that I said all this nonsense, what was I thinking? etc. |
| 16:32 |
|
skay |
at elast it isn't profound |
| 16:33 |
|
skay |
etc |
| 16:33 |
|
skay |
man, I am going to have an unhappy boss if I don't have something pretty to demo by tomorrow |
| 16:33 |
|
skay |
I should stop thinking about stuff other than doing that |
| 16:35 |
|
skay |
btw, before I go, http://pythonhosted.org/Sumatra/index.html and http://pyvideo.org/video/2039/using-sumatra-to-manage-numerical-simulations-sc |
| 16:35 |
|
pdurbin |
skay: sorry, was fighting with our designer. will catch up in a bit |
| 17:29 |
|
pdurbin |
skay: I still haven't caught up but wanted to link you to this: Git can facilitate greater reproducibility and increased transparency in science « Berkeley Initiative for Transparency in the Social Sciences - http://bitss.org/2014/03/12/git-reproducibility-transparency/ |
| 17:32 |
|
skay |
pdurbin: I am not catching up yet, but speaking of interesting blog posts and the fact that you mentioned interest in docker here is: http://bcbio.wordpress.com/2014/03/06/improving-reproducibility-and-installation-of-genomic-analysis-pipelines-with-docker/ |
| 17:33 |
|
skay |
arg why do people post links to pdfs |
| 17:33 |
|
skay |
why don't people who want to share information share it in a web friendly form as well as a pdf? |
| 17:34 |
|
* skay |
goes back afk |
| 17:34 |
|
|
LyndsySimon joined #dvn |
| 17:35 |
|
pdurbin |
javaeebot: lucky beyond the pdf conference |
| 17:35 |
|
javaeebot |
pdurbin: http://www.force11.org/beyondthepdf2 |
| 17:40 |
|
|
axfelix joined #dvn |
| 19:22 |
|
|
LyndsySimon joined #dvn |
| 19:47 |
|
|
LyndsySimon joined #dvn |
| 20:37 |
|
|
LyndsySimon joined #dvn |
| 22:14 |
|
|
axfelix joined #dvn |
| 23:37 |
|
|
garnett joined #dvn |
