Time
S
Nick
Message
00:53 axfelix joined #dataverse
03:35 axfelix joined #dataverse
05:51 zamro joined #dataverse
06:08 zamro left #dataverse
08:45 bencomp joined #dataverse
15:26 pdurbin
ok I guess my slides are good enough. talk in 90 minutes: 2016-01-12 Dataverse API Testing - http://bl.ocks.org/pdurbin/raw/814fd29916749523db9a
16:00 pameyer
good luck with the talk
16:04 axfelix joined #dataverse
16:33 metamattj joined #dataverse
16:43 axfelix joined #dataverse
17:01 pdurbin
thanks
18:25 pdurbin
talk went well. some tests failed, which was a surprise: https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-4.2.3/edu.harvard.iq$dataverse/24/testReport/
18:27 garnett joined #dataverse
18:56 pdurbin
probably a transient error. the next build passed (no code changes): https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-4.2.3/25/
19:04 pdurbin
pameyer: did you figure out which datasets you want me to try on our demo site
19:23 pameyer
pdurbin: partly - I'll send that along with login info (and my notes on the minimal testing I've done so far) once I'm done with figures and revisions
19:25 pdurbin
pameyer: can you please give me one to start with?
19:26 pdurbin
I don't *think* I'll need any login info. All the data is public.
19:28 pameyer
I was thinking login info for our dataverse
19:28 pameyer
but I'd forgotten that if you're using the demo site, you won't need that
19:28 pdurbin
right. I'll be using https://demo.dataverse.org
19:29 pdurbin
hmm. local disk only has 12 GB available though
19:30 pdurbin
pameyer: think we can get a dataset to fit into that?
19:32 pameyer
https://data.sbgrid.org/dataset/137/ is the smallest
19:32 pameyer
that's only 126M
19:32 pameyer
it uploaded with minimal problems
19:33 pdurbin
sweet. I'll start with that one if that's ok
19:33 pameyer
https://data.sbgrid.org/dataset/78 , 187, 190, 131 are the biggest ones at the moment
19:33 pameyer
137 is probably a good place to start - it should have the fewest issues
19:34 pameyer
directory structure isn't an issue for 137 (it will be for others)
19:35 pameyer
file naming is a problem w\ 137 ; and UI oddness and CPU usaged during upload too
19:35 pdurbin
fun :)
19:35 pdurbin
pameyer: do you have the ORCID ID for the lab for 137?
19:37 pameyer
let's find out...
19:41 pameyer
we don't currently have ORCID for that lab
19:41 pameyer
at least, not where we're supposed to have it
19:41 pdurbin
ok. no worries. I was thinking about using it as the "alias" of the dataverse
19:42 pameyer
good thinking :)
20:06 pdurbin
pameyer: interestingly. I'm blocked on logging in at the moment because I disabled shibboleth on that box (because I renamed the hostname)
20:16 pdurbin
pameyer: where does the image at https://data.sbgrid.org/upload/thumbnails/137/Picture1.png come from?
20:21 garnett joined #dataverse
20:23 pameyer
pdurbin: depositors can upload them during dataset registration
20:23 pameyer
if not, we either generate one from the model or use a default
20:24 pdurbin
interesting. like how we make a thumbnail from PDFs
20:25 pameyer
very similar. our depositors don't upload the model though
20:29 pdurbin
first pain point is me putting the dataset in the wrong dataverse since you can't move it
20:34 pdurbin
pameyer: not sure what dataverse category to use for a lab. I guess I'll use "Researcher".
20:34 pameyer
I thought there was a "Research Group", but I could be wrong
20:36 bencomp joined #dataverse
20:36 pdurbin
pameyer: here are the choices: http://i.imgur.com/Mt2VQuz.png
20:37 pameyer
so the second pain point could be a new dataverse category
20:38 pdurbin
pameyer: we should be writing these down
20:40 bencomp
woah, what happened to https://github.com/IQSS/dataverse/issues ? nearly 100 issues closed in a few hours!
20:44 pameyer
pdurbin: we should be writing them down
20:44 pdurbin
pameyer: beyond the logs captured by iqlogbot I mean :)
20:45 pdurbin
bencomp: at this rate we'll have zero open issues very soon! ;)
20:45 pameyer
bencomp: wonder if you can script closing github issues
20:46 pameyer
pdurbin: I was taking notes in a text file last week - haven't been doing that today
20:46 pdurbin
pameyer: I'll just start a Google Doc if you don't mind.
20:47 bencomp
pameyer: GitHub has an API and integrates with other tools for many things, including issue management
20:47 bencomp
so scripting it should be possible too
20:47 pameyer
google doc works
20:47 pdurbin
bencomp pameyer definitely. we did some issue migration with https://github.com/IQSS/redmine2github
20:48 pameyer
but if it was a few hours, it's not likely that someone did the equivalent of "github close issues/*" ;)
20:51 bencomp
somebody is (or some bodies are) really going for it
20:53 pdurbin
bencomp: these are old tickets from beta 14 and 15. I can forward you the list if you want.
20:54 bencomp
pdurbin: thanks, I'll just believe you
20:54 pdurbin
bencomp: here's the list if you're curious: https://paste.fedoraproject.org/309990/
20:55 bencomp
I got some email notifications about issues in the 1000 range :)
20:56 bencomp
I do wonder if QA is performed on these issues?
20:57 pdurbin
I didn't do any QA of them.
20:58 bencomp
it's usually kcondon who adds "tested - works, closing" as a message when he closes them
20:59 pdurbin
yep
21:01 pdurbin
pameyer: here's the doc I just started: https://docs.google.com/a/harvard.edu/document/d/1mFmemRHzCIkvoX4HJ-MKGjqnFy0sGZGjfU0oWcNt10w/edit?usp=sharing
21:04 pameyer
pdurbin: I can edit that one too
21:04 pdurbin
sure. go for it
21:11 pameyer
ah - just meant that edit worked....
21:12 pdurbin
oh. ok. either way
21:16 pdurbin
pameyer: I think the files uploaded ok (I used a zip file): https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.5072/FK2/MCHPCM
21:16 pameyer
pdurbin: I'll put stuff in - but probably not today
21:16 pdurbin
no problem
21:16 pameyer
for 137, it works with individual files too
21:17 pameyer
works = works, but the GUI sometimes reports that files are still being transferred when they're done already
21:20 pdurbin
hmm. I didn't observe that but I believe you
21:20 pdurbin
(I'm writing down questions and observations.)
21:31 pameyer
good plan - that's what I was doing too. then I realized my notes probably wouldn't make sense to anybody else; aka - why I needed a plan
21:33 pdurbin
heh
21:34 pdurbin
pameyer: can you give me a second dataset to work on? I think going to pass 137 to the metadata experts
21:35 pdurbin
by tomorrow morning, I mean. I typically leave to pick up my kids at 4:30 (now). I'll check the logs
21:50 pameyer
pdurbin: try https://data.sbgrid.org/dataset/5/ (15G)
21:52 pameyer
pdurbin: these sizes are on zfs (aka - w\ compression), so they might be larger in the demo site
22:15 pdurbin
pameyer: hmm. 15 GB won't fit in 12 GB of free disk space. Have anything smaller?
22:18 pameyer
pdurbin: it *better* not be tomorrow yet
22:18 pameyer
and good catch on the size...
22:19 bencomp
oops, I created https://github.com/IQSS/dataverse/pull/2860 ;)
22:20 pameyer
https://data.sbgrid.org/dataset/24 ; ~512MB
22:24 pameyer
bencomp: at least you didn't try to get pdurbin to put > 15GB of data into 12 GB of free space...
22:30 bencomp
pameyer: no, I tend to put a lot of effort in "code aesthetics" that mostly only I care about
22:31 bencomp
and sometimes the effort is ignored
23:25 axfelix joined #dataverse
23:58 pdurbin
pameyer: 24 it is. thanks
23:58 pdurbin
we *talk* about pull requests a lot :)
