IQSS logo

IRC log for #dvn, 2014-01-21

We've moved! Please join #dataverse instead. The new logs are at http://irclog.iq.harvard.edu/dataverse/today

| Channels | #dvn index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
00:54 axfelix joined #dvn
01:55 axfelix joined #dvn
16:07 axfelix joined #dvn
16:47 Robert_Liebowitz joined #dvn
16:51 Robert_Liebowitz I'm having some trouble with the API for downloading a file off the Dataverse. Is there documentation for this?
17:25 axfelix pdurbin: sorry, just saw this now, but yup, it's been an ongoing testing issue
17:25 axfelix we're holding out hope that it's still the source of OCUL's troubles since we haven't been able to isolate anything else
17:32 skay axfelix: the service document endpoint is returning a different href than it has a certificate for
17:33 axfelix skay: we've tried the endpoint that has the correct cert too with not much more luck
17:33 skay axfelix: so it is not a problem with the client code as such
17:33 skay axfelix: do you mean accounts on the dvn-1 service?
17:33 skay oops, I meant the dvn-4 service
17:34 axfelix skay: nope, we've only used dvn-demo and dvn-1
17:35 skay okay, well, the dataverse service-document returned by dvn-1 is returning inaccurate information.
17:35 skay it returns dvn-demo.iq.harvard.edu regardless
17:35 axfelix thanks, you might've just saved us a lot of pain ...
17:35 skay so then the sword client
17:36 pdurbin Robert_Liebowitz: you want to download the file itself and not the metadata for the study (author, title, etc.)?
17:36 skay http://dpaste.com/hold/1561595/ http://dpaste.com/hold/1561612/ oh my goodness hte pain
17:36 skay :)
17:36 skay I spent all yesterday feeling like an idiot :)
17:36 skay so, if you make an acct on dvn-4 and test against that you should be okay
17:37 skay I think maybe http://dpaste.com/hold/1561595/ illustrates it better than the other
17:37 axfelix fantastic. will report back.
17:38 skay you make a Connection as done in the dvn_client example. that connection wraps connections from the sword2 package, and sword2 talks to the dataverse service -- the service is returning a different answer than the real(?) hostname
17:38 skay basename, not sure of the proper vocab
17:39 skay yeah! definitely try it out. I hope it works
17:39 skay It worked fo rme
17:39 skay you can even try mutating the workspace that comes back from the swordConnection to set the href by hand just to check
17:39 skay if you want to still try with dvn-1
17:40 skay I haven't looked at the java code to see how that service populates the information
17:41 skay I need to work on the python side to try and see if I can get a demo up or at least a test to show my boss
17:41 skay :)
17:41 axfelix nice to see a python example actually
17:42 skay axfelix: are you located in the Boston area (the project is Harvard dataverse, right?  but I wasn't sure if everyone works at Harvard or this is a collaboration among a lot of different places)
17:43 axfelix skay: nope, I'm in vancouver, with simon fraser university, we're working on a dataverse plugin for open journal systems
17:43 skay I ask because I suggested to OpenHatch that they consider contacting people for the Open Source Comes to Campus workshops they do throughout the year
17:43 skay Oh cool! are you going to pycon? I want to run a sprint for my project, but I think perhaps a collection of open science sprints would be neat
17:44 skay anyway, but what is cool is that the openhatch series of events is not python specific, it's more about teaching students about open source and helping them become open source contributors
17:45 axfelix montreal's a little far for me, particularly seeing as I'm not a FT developer
17:45 skay and I know one of the persons in charge of the iniative and she is active in the open science community
17:45 axfelix but hey, you never know
17:46 skay maybe you could apply for a grant from someone? Mozilla has a science lab, and Greg Wilson lives in Toronto which I know is also far, but hey, Canada!
17:46 skay (I'm not joking, I like Toronto. I live in Chicago and once took a road trip from Chicago through Toronto and then on through New Foundland)
17:46 skay (I would like to go in the other direction as well)
17:47 skay Perhaps you could apply to do the software-carpentry workshop during the sprints?
17:47 skay I don't know if they have grants for that, but who knows
17:47 skay sorry for derailing the channel with off topic information. I am a bit obsessed with trying to have a big open source community for science related concerns
17:48 axfelix no worries, I'll look into it :)
17:50 posixeleni joined #dvn
17:57 Robert_Liebowitz pdurbin: Yeah I'm looking to download the file itself. I'm using the python client, and it looks like sword wants a content iri from the deposit receipt, which it's not giving me.
17:58 Robert_Liebowitz I've also tried some variations of using the download link that the website provides, but I can't pass it credentials.
17:59 pdurbin Robert_Liebowitz: unfortunately, I ran out of time to implement the actual downloading of the file via SWORD. If you attempt to you should get a error that suggest using our older "Data Sharing API": http://thedata.harvard.edu/guides/dataverse-api-main.html#file-access-api
18:04 skay pdurbin: how much work do you think it would be to implement that?
18:04 Robert_Liebowitz Using that I'm getting 404s with the fileId from the download link.
18:04 pdurbin Robert_Liebowitz: I seem to recall that Jeff and Josh had planned to use the download link that the website provides
18:07 pdurbin Robert_Liebowitz: I think that's what "Get download url" under files on the gameplan doc means: https://docs.google.com/document/d/1wTWhJHQajSjEb_swjc4i_ZsMRAv_RZfQjjje4W9Nj_4/edit?usp=sharing
18:08 Robert_Liebowitz I haven't been able to pass credentials. Josh and I have tried both logging in and passing a username/password through auth and URL, and it always results in a 403
18:10 pdurbin Robert_Liebowitz: do you have an example url to a file you're trying to download?
18:11 Robert_Liebowitz Here's the text file I've been trying to access: http://dvn-4.hmdc.harvard.edu/dvn/FileDownload/?fileId=2339761&xff=0&versionNumber=1
18:11 Robert_Liebowitz The download link works if I'm already logged in; otherwise, it just says I don't have permission
18:12 pdurbin "You do not have permission to download this file."
18:12 pdurbin as you said :)
18:12 pdurbin hmm
18:13 Robert_Liebowitz Right.
18:13 Robert_Liebowitz I have the username and password to let me in, but I don't have a way to pass it.
18:13 pdurbin 403 Forbidden
18:14 pdurbin what if you check for 403 Forbidden and if you are forbidden, you sent the user to the study instead so the user can log in?
18:14 pdurbin send*
18:16 Robert_Liebowitz That would work for the user that creates/owns the study, but the idea is that the user should be able to use their credentials to allow other people to access the file without giving their username and password
18:17 pdurbin Robert_Liebowitz: sure, so the user could make the file public, for example, right?
18:17 jwhitney joined #dvn
18:18 Robert_Liebowitz Right.
18:19 Robert_Liebowitz The way the OSF is set up, he could either make his files public, or share them with just the other contributors on his project
18:19 pdurbin Robert_Liebowitz: so those other contributors could sign up for dataverse accounts, right?
18:20 Robert_Liebowitz They shouldn't have to.
18:20 pdurbin ?
18:21 pdurbin permissions to download non-public files is controlled within the dataverse app
18:22 Robert_Liebowitz So the only way for anyone who isn't the original owner of those files to access non-public files is with a dataverse account that the original owner has to then approve?
18:23 pdurbin right. it's like how you have to have a github account if someone wants to give you access to a private github repo
18:23 skay Would a service like OSF or researchcompendia be able to make a service account such that members of a group on OSF, for example, could have a page on OSF/etc to view files
18:24 Robert_Liebowitz Exactly what skay said.
18:24 skay Robert_Liebowitz: yes, I think I have a similar use case
18:24 skay I was thinking I would make a user account that is actually for the researchcompendia service
18:25 skay and then have code in my app that manages credentials on behalf of my users -- they may or may not have dvn accounts
18:25 Robert_Liebowitz We have a GitHub addon, and if the owner of that repo gives us his credentials, he can then decide which users on the OSF can access the files. They don't need a GitHub account.
18:25 sbmarks joined #dvn
18:29 pdurbin Robert_Liebowitz: so this is why you want to pass the credentials of the owner of the dataverse study
18:30 Robert_Liebowitz pdurbin: Exactly. We want our users to be able to manage and share files entirely from the OSF
18:31 pdurbin interesting
18:32 sbmarks axfelix jwhitney posixeleni: no joy with dvn-4  =(
18:33 posixeleni sbmarks: /me scratches her head =(
18:34 posixeleni sbmarks: same error as before?
18:34 sbmarks sorry, I know I am problematic
18:34 pdurbin Robert_Liebowitz: can you pass credentials to the Data Sharing API? (the non-SWORD one)
18:34 Robert_Liebowitz pdurbin: With GitHub as the example of an add-on we have working, a user can log into their github account and create an access token. They can then choose to use that token to share specific repositories either with the general public or each other. They can also grant editing permissions.
18:34 posixeleni sbmarks: thanks for trying this!
18:34 skay sbmarks: do you have a traceback or something you can share in a pastebin?
18:35 skay http://dpaste.com/
18:36 Robert_Liebowitz pdurbin: I haven't been able to. Even if it worked through the URL though, it wouldn't be hard for other users to figure out a dataverse username/password from the download link.
18:37 skay Robert_Liebowitz: if you do get the u/pw link working, then could you have OSF handle that in the backend and not present the user the url in the front end?
18:38 skay the user would have an osf url, the osf framework would then get the real url and return the file? weaksauce?]
18:38 sbmarks skay: if you can tell me where I should be grabbing the traceback from. I am not seeing any errors in the httpd error log, which is where I expect to see OJS related errors
18:39 sbmarks totally unrelated question: are there .war files for 3.6.2?
18:39 skay sbmarks: I am not familiar with OJS, can you make a small file to replicate this issue in a few lines? and then run that?
18:39 skay also, I was assuming you were using the python client in your code, but if this is not the case, let me know
18:40 sbmarks I think we may be talking about two different things here
18:40 pdurbin at least two ;)
18:40 axfelix skay: OJS dataverse plugin is php, here: https://github.com/jwhitney/dataverse
18:41 pdurbin Robert_Liebowitz: we don't want people to be able to figure out anyone's password, obviously :)
18:42 skay sbmarks: oh, yes. different things. sorry. I was having a problem with cert signing for dvn-1 while using hte python client
18:42 sbmarks skay: np =)
18:43 skay log all the things! :)
18:43 pdurbin Robert_Liebowitz: what if you limited yourself to public files for now?
18:51 jwhitney sbmarks: so no dice on dvn-4.
18:51 sbmarks yeah, nope. all appearances it looks and acts the same
18:51 jwhitney sbmarks: your version of the plugin's been patched to disable host & peer verification in curl requests made by the swordapp
18:52 jwhitney which should allow requests on the dvn-demo urls returned in the servicedoc
18:52 sbmarks jwhitney: yeah, it has the patch applied that you and axfelix sent me a while back
18:53 jwhitney sbmarks: thought so, but just running through a mental checklist
18:53 sbmarks just to be crystal clear, I am pointing the OJS plugin at http://dvn-4.hmdc.harvard.edu/dvn
18:54 pdurbin jwhitney: so you're able to work around the dual hostname issue
18:54 jwhitney sbmarks: can you try https:// etc? the plugin doesn't redirect requests from http to https
18:55 jwhitney pdurbin: yes, but only by turning off host & peer verification
18:57 pdurbin jwhitney: ok, there's a new (internal) ticket to work on this: #173003: Issues Testing Data Deposit API: Real Certificate for dvn-demo ? - https://help.hmdc.harvard.edu/Ticket/Display.html?id=173003
18:59 sbmarks jwhitney: https://dvn-4.hmdc.harvard.edu/dvn is also no joy
19:04 jwhitney sbmarks: ok, thanks.
19:04 jwhitney sbmarks: have to run, meeting. back in about 30.
19:08 Robert_Liebowitz joined #dvn
19:11 Robert_Liebowitz pdurbin: I can look into just working with public files for now, although it might be a read-only situation.
19:13 pdurbin Robert_Liebowitz: ok
19:15 sbmarks since it has quieted down, is 3.6.2 available as a .war file?
19:16 pdurbin sbmarks: nope. the newest is 3.6: http://sourceforge.net/projects/dvn/files/dvn/
19:16 sbmarks but I want DOIs too =(
19:17 pdurbin :)
19:19 sbmarks do minor revisions not get released?
19:19 pdurbin sbmarks: 3.6 has DOIs, no? http://thedata.org/book/latest-releases
19:19 sbmarks i was JUST looking at that. I'm an idiot.
19:19 pdurbin :)
19:20 sbmarks I'm just so excited that we got off of 2.2.5 that now I'm drunk with power
19:20 pdurbin heh
19:20 sbmarks but apparently I'm just drunk
19:20 pdurbin sbmarks: well, minor revisions typically do get released. I added a ticket: https://redmine.hmdc.harvard.edu/issues/3424
19:21 sbmarks thanks! It's pretty low priority now that I have my head on straight
19:21 sbmarks but might as well
19:25 pdurbin sbmarks: congrats on the upgrade
19:25 sbmarks pdurbin: thanks. i feel like I passed a kidney stone
19:30 pdurbin heh
20:49 posixeleni joined #dvn
21:23 skay pdurbin: any reason to keep dvn_client files nested in src?
21:29 skay Im thinking about helping get dvn_client packaged, so would like to have it looking more like a python repo
21:29 skay but that might not be helpful for you, so I can also not do that task
22:30 pdurbin skay: however you want to organize it is fine. I don't know Python very well
22:38 skay pdurbin: thanks. I wanted to check before asking to change directory structure
22:39 skay it can be a pain
22:51 pdurbin sure. makes sense
23:31 axfelix joined #dvn

| Channels | #dvn index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

We've moved! Please join #dataverse instead. The new logs are at http://irclog.iq.harvard.edu/dataverse/today