| Time |
S |
Nick |
Message |
| 00:54 |
|
|
axfelix joined #dvn |
| 01:55 |
|
|
axfelix joined #dvn |
| 16:07 |
|
|
axfelix joined #dvn |
| 16:47 |
|
|
Robert_Liebowitz joined #dvn |
| 16:51 |
|
Robert_Liebowitz |
I'm having some trouble with the API for downloading a file off the Dataverse. Is there documentation for this? |
| 17:25 |
|
axfelix |
pdurbin: sorry, just saw this now, but yup, it's been an ongoing testing issue |
| 17:25 |
|
axfelix |
we're holding out hope that it's still the source of OCUL's troubles since we haven't been able to isolate anything else |
| 17:32 |
|
skay |
axfelix: the service document endpoint is returning a different href than it has a certificate for |
| 17:33 |
|
axfelix |
skay: we've tried the endpoint that has the correct cert too with not much more luck |
| 17:33 |
|
skay |
axfelix: so it is not a problem with the client code as such |
| 17:33 |
|
skay |
axfelix: do you mean accounts on the dvn-1 service? |
| 17:33 |
|
skay |
oops, I meant the dvn-4 service |
| 17:34 |
|
axfelix |
skay: nope, we've only used dvn-demo and dvn-1 |
| 17:35 |
|
skay |
okay, well, the dataverse service-document returned by dvn-1 is returning inaccurate information. |
| 17:35 |
|
skay |
it returns dvn-demo.iq.harvard.edu regardless |
| 17:35 |
|
axfelix |
thanks, you might've just saved us a lot of pain ... |
| 17:35 |
|
skay |
so then the sword client |
| 17:36 |
|
pdurbin |
Robert_Liebowitz: you want to download the file itself and not the metadata for the study (author, title, etc.)? |
| 17:36 |
|
skay |
http://dpaste.com/hold/1561595/ http://dpaste.com/hold/1561612/ oh my goodness hte pain |
| 17:36 |
|
skay |
:) |
| 17:36 |
|
skay |
I spent all yesterday feeling like an idiot :) |
| 17:36 |
|
skay |
so, if you make an acct on dvn-4 and test against that you should be okay |
| 17:37 |
|
skay |
I think maybe http://dpaste.com/hold/1561595/ illustrates it better than the other |
| 17:37 |
|
axfelix |
fantastic. will report back. |
| 17:38 |
|
skay |
you make a Connection as done in the dvn_client example. that connection wraps connections from the sword2 package, and sword2 talks to the dataverse service -- the service is returning a different answer than the real(?) hostname |
| 17:38 |
|
skay |
basename, not sure of the proper vocab |
| 17:39 |
|
skay |
yeah! definitely try it out. I hope it works |
| 17:39 |
|
skay |
It worked fo rme |
| 17:39 |
|
skay |
you can even try mutating the workspace that comes back from the swordConnection to set the href by hand just to check |
| 17:39 |
|
skay |
if you want to still try with dvn-1 |
| 17:40 |
|
skay |
I haven't looked at the java code to see how that service populates the information |
| 17:41 |
|
skay |
I need to work on the python side to try and see if I can get a demo up or at least a test to show my boss |
| 17:41 |
|
skay |
:) |
| 17:41 |
|
axfelix |
nice to see a python example actually |
| 17:42 |
|
skay |
axfelix: are you located in the Boston area (the project is Harvard dataverse, right? but I wasn't sure if everyone works at Harvard or this is a collaboration among a lot of different places) |
| 17:43 |
|
axfelix |
skay: nope, I'm in vancouver, with simon fraser university, we're working on a dataverse plugin for open journal systems |
| 17:43 |
|
skay |
I ask because I suggested to OpenHatch that they consider contacting people for the Open Source Comes to Campus workshops they do throughout the year |
| 17:43 |
|
skay |
Oh cool! are you going to pycon? I want to run a sprint for my project, but I think perhaps a collection of open science sprints would be neat |
| 17:44 |
|
skay |
anyway, but what is cool is that the openhatch series of events is not python specific, it's more about teaching students about open source and helping them become open source contributors |
| 17:45 |
|
axfelix |
montreal's a little far for me, particularly seeing as I'm not a FT developer |
| 17:45 |
|
skay |
and I know one of the persons in charge of the iniative and she is active in the open science community |
| 17:45 |
|
axfelix |
but hey, you never know |
| 17:46 |
|
skay |
maybe you could apply for a grant from someone? Mozilla has a science lab, and Greg Wilson lives in Toronto which I know is also far, but hey, Canada! |
| 17:46 |
|
skay |
(I'm not joking, I like Toronto. I live in Chicago and once took a road trip from Chicago through Toronto and then on through New Foundland) |
| 17:46 |
|
skay |
(I would like to go in the other direction as well) |
| 17:47 |
|
skay |
Perhaps you could apply to do the software-carpentry workshop during the sprints? |
| 17:47 |
|
skay |
I don't know if they have grants for that, but who knows |
| 17:47 |
|
skay |
sorry for derailing the channel with off topic information. I am a bit obsessed with trying to have a big open source community for science related concerns |
| 17:48 |
|
axfelix |
no worries, I'll look into it :) |
| 17:50 |
|
|
posixeleni joined #dvn |
| 17:57 |
|
Robert_Liebowitz |
pdurbin: Yeah I'm looking to download the file itself. I'm using the python client, and it looks like sword wants a content iri from the deposit receipt, which it's not giving me. |
| 17:58 |
|
Robert_Liebowitz |
I've also tried some variations of using the download link that the website provides, but I can't pass it credentials. |
| 17:59 |
|
pdurbin |
Robert_Liebowitz: unfortunately, I ran out of time to implement the actual downloading of the file via SWORD. If you attempt to you should get a error that suggest using our older "Data Sharing API": http://thedata.harvard.edu/guides/dataverse-api-main.html#file-access-api |
| 18:04 |
|
skay |
pdurbin: how much work do you think it would be to implement that? |
| 18:04 |
|
Robert_Liebowitz |
Using that I'm getting 404s with the fileId from the download link. |
| 18:04 |
|
pdurbin |
Robert_Liebowitz: I seem to recall that Jeff and Josh had planned to use the download link that the website provides |
| 18:07 |
|
pdurbin |
Robert_Liebowitz: I think that's what "Get download url" under files on the gameplan doc means: https://docs.google.com/document/d/1wTWhJHQajSjEb_swjc4i_ZsMRAv_RZfQjjje4W9Nj_4/edit?usp=sharing |
| 18:08 |
|
Robert_Liebowitz |
I haven't been able to pass credentials. Josh and I have tried both logging in and passing a username/password through auth and URL, and it always results in a 403 |
| 18:10 |
|
pdurbin |
Robert_Liebowitz: do you have an example url to a file you're trying to download? |
| 18:11 |
|
Robert_Liebowitz |
Here's the text file I've been trying to access: http://dvn-4.hmdc.harvard.edu/dvn/FileDownload/?fileId=2339761&xff=0&versionNumber=1 |
| 18:11 |
|
Robert_Liebowitz |
The download link works if I'm already logged in; otherwise, it just says I don't have permission |
| 18:12 |
|
pdurbin |
"You do not have permission to download this file." |
| 18:12 |
|
pdurbin |
as you said :) |
| 18:12 |
|
pdurbin |
hmm |
| 18:13 |
|
Robert_Liebowitz |
Right. |
| 18:13 |
|
Robert_Liebowitz |
I have the username and password to let me in, but I don't have a way to pass it. |
| 18:13 |
|
pdurbin |
403 Forbidden |
| 18:14 |
|
pdurbin |
what if you check for 403 Forbidden and if you are forbidden, you sent the user to the study instead so the user can log in? |
| 18:14 |
|
pdurbin |
send* |
| 18:16 |
|
Robert_Liebowitz |
That would work for the user that creates/owns the study, but the idea is that the user should be able to use their credentials to allow other people to access the file without giving their username and password |
| 18:17 |
|
pdurbin |
Robert_Liebowitz: sure, so the user could make the file public, for example, right? |
| 18:17 |
|
|
jwhitney joined #dvn |
| 18:18 |
|
Robert_Liebowitz |
Right. |
| 18:19 |
|
Robert_Liebowitz |
The way the OSF is set up, he could either make his files public, or share them with just the other contributors on his project |
| 18:19 |
|
pdurbin |
Robert_Liebowitz: so those other contributors could sign up for dataverse accounts, right? |
| 18:20 |
|
Robert_Liebowitz |
They shouldn't have to. |
| 18:20 |
|
pdurbin |
? |
| 18:21 |
|
pdurbin |
permissions to download non-public files is controlled within the dataverse app |
| 18:22 |
|
Robert_Liebowitz |
So the only way for anyone who isn't the original owner of those files to access non-public files is with a dataverse account that the original owner has to then approve? |
| 18:23 |
|
pdurbin |
right. it's like how you have to have a github account if someone wants to give you access to a private github repo |
| 18:23 |
|
skay |
Would a service like OSF or researchcompendia be able to make a service account such that members of a group on OSF, for example, could have a page on OSF/etc to view files |
| 18:24 |
|
Robert_Liebowitz |
Exactly what skay said. |
| 18:24 |
|
skay |
Robert_Liebowitz: yes, I think I have a similar use case |
| 18:24 |
|
skay |
I was thinking I would make a user account that is actually for the researchcompendia service |
| 18:25 |
|
skay |
and then have code in my app that manages credentials on behalf of my users -- they may or may not have dvn accounts |
| 18:25 |
|
Robert_Liebowitz |
We have a GitHub addon, and if the owner of that repo gives us his credentials, he can then decide which users on the OSF can access the files. They don't need a GitHub account. |
| 18:25 |
|
|
sbmarks joined #dvn |
| 18:29 |
|
pdurbin |
Robert_Liebowitz: so this is why you want to pass the credentials of the owner of the dataverse study |
| 18:30 |
|
Robert_Liebowitz |
pdurbin: Exactly. We want our users to be able to manage and share files entirely from the OSF |
| 18:31 |
|
pdurbin |
interesting |
| 18:32 |
|
sbmarks |
axfelix jwhitney posixeleni: no joy with dvn-4 =( |
| 18:33 |
|
posixeleni |
sbmarks: /me scratches her head =( |
| 18:34 |
|
posixeleni |
sbmarks: same error as before? |
| 18:34 |
|
sbmarks |
sorry, I know I am problematic |
| 18:34 |
|
pdurbin |
Robert_Liebowitz: can you pass credentials to the Data Sharing API? (the non-SWORD one) |
| 18:34 |
|
Robert_Liebowitz |
pdurbin: With GitHub as the example of an add-on we have working, a user can log into their github account and create an access token. They can then choose to use that token to share specific repositories either with the general public or each other. They can also grant editing permissions. |
| 18:34 |
|
posixeleni |
sbmarks: thanks for trying this! |
| 18:34 |
|
skay |
sbmarks: do you have a traceback or something you can share in a pastebin? |
| 18:35 |
|
skay |
http://dpaste.com/ |
| 18:36 |
|
Robert_Liebowitz |
pdurbin: I haven't been able to. Even if it worked through the URL though, it wouldn't be hard for other users to figure out a dataverse username/password from the download link. |
| 18:37 |
|
skay |
Robert_Liebowitz: if you do get the u/pw link working, then could you have OSF handle that in the backend and not present the user the url in the front end? |
| 18:38 |
|
skay |
the user would have an osf url, the osf framework would then get the real url and return the file? weaksauce?] |
| 18:38 |
|
sbmarks |
skay: if you can tell me where I should be grabbing the traceback from. I am not seeing any errors in the httpd error log, which is where I expect to see OJS related errors |
| 18:39 |
|
sbmarks |
totally unrelated question: are there .war files for 3.6.2? |
| 18:39 |
|
skay |
sbmarks: I am not familiar with OJS, can you make a small file to replicate this issue in a few lines? and then run that? |
| 18:39 |
|
skay |
also, I was assuming you were using the python client in your code, but if this is not the case, let me know |
| 18:40 |
|
sbmarks |
I think we may be talking about two different things here |
| 18:40 |
|
pdurbin |
at least two ;) |
| 18:40 |
|
axfelix |
skay: OJS dataverse plugin is php, here: https://github.com/jwhitney/dataverse |
| 18:41 |
|
pdurbin |
Robert_Liebowitz: we don't want people to be able to figure out anyone's password, obviously :) |
| 18:42 |
|
skay |
sbmarks: oh, yes. different things. sorry. I was having a problem with cert signing for dvn-1 while using hte python client |
| 18:42 |
|
sbmarks |
skay: np =) |
| 18:43 |
|
skay |
log all the things! :) |
| 18:43 |
|
pdurbin |
Robert_Liebowitz: what if you limited yourself to public files for now? |
| 18:51 |
|
jwhitney |
sbmarks: so no dice on dvn-4. |
| 18:51 |
|
sbmarks |
yeah, nope. all appearances it looks and acts the same |
| 18:51 |
|
jwhitney |
sbmarks: your version of the plugin's been patched to disable host & peer verification in curl requests made by the swordapp |
| 18:52 |
|
jwhitney |
which should allow requests on the dvn-demo urls returned in the servicedoc |
| 18:52 |
|
sbmarks |
jwhitney: yeah, it has the patch applied that you and axfelix sent me a while back |
| 18:53 |
|
jwhitney |
sbmarks: thought so, but just running through a mental checklist |
| 18:53 |
|
sbmarks |
just to be crystal clear, I am pointing the OJS plugin at http://dvn-4.hmdc.harvard.edu/dvn |
| 18:54 |
|
pdurbin |
jwhitney: so you're able to work around the dual hostname issue |
| 18:54 |
|
jwhitney |
sbmarks: can you try https:// etc? the plugin doesn't redirect requests from http to https |
| 18:55 |
|
jwhitney |
pdurbin: yes, but only by turning off host & peer verification |
| 18:57 |
|
pdurbin |
jwhitney: ok, there's a new (internal) ticket to work on this: #173003: Issues Testing Data Deposit API: Real Certificate for dvn-demo ? - https://help.hmdc.harvard.edu/Ticket/Display.html?id=173003 |
| 18:59 |
|
sbmarks |
jwhitney: https://dvn-4.hmdc.harvard.edu/dvn is also no joy |
| 19:04 |
|
jwhitney |
sbmarks: ok, thanks. |
| 19:04 |
|
jwhitney |
sbmarks: have to run, meeting. back in about 30. |
| 19:08 |
|
|
Robert_Liebowitz joined #dvn |
| 19:11 |
|
Robert_Liebowitz |
pdurbin: I can look into just working with public files for now, although it might be a read-only situation. |
| 19:13 |
|
pdurbin |
Robert_Liebowitz: ok |
| 19:15 |
|
sbmarks |
since it has quieted down, is 3.6.2 available as a .war file? |
| 19:16 |
|
pdurbin |
sbmarks: nope. the newest is 3.6: http://sourceforge.net/projects/dvn/files/dvn/ |
| 19:16 |
|
sbmarks |
but I want DOIs too =( |
| 19:17 |
|
pdurbin |
:) |
| 19:19 |
|
sbmarks |
do minor revisions not get released? |
| 19:19 |
|
pdurbin |
sbmarks: 3.6 has DOIs, no? http://thedata.org/book/latest-releases |
| 19:19 |
|
sbmarks |
i was JUST looking at that. I'm an idiot. |
| 19:19 |
|
pdurbin |
:) |
| 19:20 |
|
sbmarks |
I'm just so excited that we got off of 2.2.5 that now I'm drunk with power |
| 19:20 |
|
pdurbin |
heh |
| 19:20 |
|
sbmarks |
but apparently I'm just drunk |
| 19:20 |
|
pdurbin |
sbmarks: well, minor revisions typically do get released. I added a ticket: https://redmine.hmdc.harvard.edu/issues/3424 |
| 19:21 |
|
sbmarks |
thanks! It's pretty low priority now that I have my head on straight |
| 19:21 |
|
sbmarks |
but might as well |
| 19:25 |
|
pdurbin |
sbmarks: congrats on the upgrade |
| 19:25 |
|
sbmarks |
pdurbin: thanks. i feel like I passed a kidney stone |
| 19:30 |
|
pdurbin |
heh |
| 20:49 |
|
|
posixeleni joined #dvn |
| 21:23 |
|
skay |
pdurbin: any reason to keep dvn_client files nested in src? |
| 21:29 |
|
skay |
Im thinking about helping get dvn_client packaged, so would like to have it looking more like a python repo |
| 21:29 |
|
skay |
but that might not be helpful for you, so I can also not do that task |
| 22:30 |
|
pdurbin |
skay: however you want to organize it is fine. I don't know Python very well |
| 22:38 |
|
skay |
pdurbin: thanks. I wanted to check before asking to change directory structure |
| 22:39 |
|
skay |
it can be a pain |
| 22:51 |
|
pdurbin |
sure. makes sense |
| 23:31 |
|
|
axfelix joined #dvn |
