IQSS logo

IRC log for #dataverse, 2015-09-24

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
05:25 sivoais joined #dataverse
05:35 sivoais joined #dataverse
05:45 sivoais joined #dataverse
07:55 bencomp joined #dataverse
09:53 romainM are you going to make a dataset printing function soon ?
09:53 romainM (a pdf export, for example ?)
10:18 pdurbin romainM: is the problem that the dataset doesn't print very well from a browser? (I haven't tried)
10:18 pdurbin I searched for "PDF" in our issue tracker and found this: Make a PDF or HTML version of the DDI Codebook XML · Issue #739 · IQSS/dataverse - https://github.com/IQSS/dataverse/issues/739
10:18 pdurbin romainM: does that sounds like what you want?
11:06 romainM sorry, I was having lunch :)
11:07 romainM something like that, yep
11:10 romainM the browser printing is sometimes buggy
11:10 romainM and not "beautiful" :D
11:10 romainM (try to print this one : https://beta.dataverse.org/dataset.xhtml?persistentId=doi:10.5072/FK2/BWSZDW)
11:27 bencomp pdurbin: you were working on roundtripping DDI, right?
11:29 bencomp what do you think of https://github.com/IQSS/dataverse/blob/4.2/src/main/java/edu/harvard/iq/dataverse/api/imports/ImportDDIServiceBean.java#L1589 ?
11:30 pdurbin bencomp: you're pointing out a type safety problem, right?
11:31 bencomp yes :)
11:31 bencomp the code isn't actually called, but it's a TODO to add it back in
11:32 bencomp it is the exact same problem I pointed out in #775
11:32 pdurbin yep, I understand
11:33 pdurbin here's the start of my idea for trying to roundtrip DDI: add test framework for exporting datasets as DDI #2579 · IQSS/dataverse@49dd844 - https://github.com/IQSS/dataverse/commit/49dd8445a5830f03b1ab6e4c7ea5f8c6e42ac49a
11:33 petrichor joined #dataverse
11:34 pdurbin romainM: hmm. from ubuntu anyway, when I print to PDF that dataset looks pretty terrible. text overlaps and it's unreadable
11:35 romainM I'm on ubuntu too, and that's exactly what I had :D
11:37 bencomp pdurbin: great work on the integration test
11:41 pdurbin romainM: can you please take a screenshot and add it to that issue?
11:42 pdurbin bencomp: thanks. if you have any DDI files I can test with (public, that I can add to the code base) please let me know
11:42 pdurbin now I'd better get my kids to school!
11:48 romainM to the issue you were talking about ?
11:48 romainM or I add a new issue ?
11:49 romainM (I guess he's already gone :D)
12:12 pdurbin romainM: hmm, I think a new issue would be better
12:20 romainM ok, I will do that
12:23 romainM I don't know what title to put ... Page printing: Messy printing of dataset ?
12:25 pdurbin romainM: a picture is worth a thousand words :)
12:26 romainM I will put the screen, np, I was just looking for a good title ^^
12:28 romainM done
12:32 * romainM has to go and will be back later
12:58 pdurbin romainM: I just tried it from Mac (I'm at work now) and it looks equally messy when I try to print a dataset. Thanks for opening https://github.com/IQSS/dataverse/issues/2584
14:14 bencomp woohoo, I successfully converted my normal account to a shib account in our acceptance environment
14:15 pdurbin bencomp: great news!
14:18 bencomp next step: shib groups... #1515
14:19 * pdurbin looks at Shibboleth Groups: arbitrary attributes and regex support · Issue #1515 · IQSS/dataverse - https://github.com/IQSS/dataverse/issues/1515
14:19 pdurbin bencomp: that hasn't been implemented yet
14:19 bencomp pdurbin: I know :(
14:19 pdurbin bencomp: can you make use of this instead? Already implemented: Shibboleth: Institution-wide Shibboleth groups · Issue #1401 · IQSS/dataverse - https://github.com/IQSS/dataverse/issues/1401
14:21 bencomp well, no, we still have that single IdP
14:21 bencomp meaning users logging in through that IdP federation could come from virtually any institution
14:22 pdurbin right. a single IdP. hmm
14:23 pdurbin we plan to support all the InCommon IdPs for the Harvard Dataverse: https://incommon.org/federation/info/all-entities.html#IdPs
14:27 bencomp we plan to support all the SURFconext IdPs, but in the SURFconext Dashboard each has to allow personal information to be shared
14:27 bencomp https://wiki.surfnet.nl/display/conextsupport/List+of+Connected+Identity+Providers
14:32 bencomp oh, and as these institutions must pay before their users get permission to add data, we need a reliable way to identify users' institutions
14:34 pdurbin bencomp: I'm confused. I thought you said you'll only have one IdP.
14:37 pdurbin bencomp: can you please write some more about all this in the Google Doc you linked from https://github.com/IQSS/dataverse/issues/2548 ?
14:42 bencomp pdurbin: as an SP, we connect with 1 IdP: SURFconext. That is a federation that acts as a hub, providing a Where Are You From page and forwarding users to the actual institutional IdPs
14:43 bencomp does it help to think of it as a SAML bus?
14:43 bencomp :)
14:49 pdurbin is the bus going to hit me? I ride my bike to work every day so I worry about these things
14:51 Abhi joined #dataverse
14:52 pdurbin bencomp: I'd like to know what the IdP values are before the pipe ("|") in the persistentuserid column of your authenticateduserlookup table: https://github.com/IQSS/dataverse/issues/1151#issuecomment-72555357 . Are they all the same? Will they be different? For the Harvard Dataverse we expect to have values like https://fed.huit.harvard.edu/idp/shibboleth and urn:mace:incommon:mit.edu etc.
14:53 pdurbin which you can also see at https://github.com/IQSS/dataverse/blob/v4.1/scripts/api/data/shibGroupHarvard.json and https://github.com/IQSS/dataverse/blob/v4.1/scripts/api/data/shibGroupMit.json
14:59 pdurbin bencomp: switching gears for a second, check out how https://github.com/keycloak/keycloak is using Swagger to generate API docs per http://blog.keycloak.org/2015/09/having-fun-with-rest-api-documentation.html and https://plus.google.com/110111685369300863074/posts/apL1iC9GpWi
15:27 bencomp https://engine.surfconext.nl/authentication/idp/metadata is the IdP value
15:27 bencomp pdurbin: ^
15:27 bencomp also: interesting story about Swagger
15:30 pdurbin bencomp: so you don't anticipate having any other IdP values in your authenticateduserlookup table?
15:31 bencomp pdurbin: indeed
15:32 pdurbin bencomp: ok, so we agree that you'll have a single IdP. Which means you'll get no value from the Institution-wide Shibboleth groups we implemented in https://github.com/IQSS/dataverse/issues/1401 :(
15:33 pdurbin bencomp: it looks like you told me this in February: https://github.com/IQSS/dataverse/issues/1401#issuecomment-73018520
15:33 bencomp correct
15:34 pdurbin what you want is this issue but it doesn't even have a milestone: Shibboleth Groups: arbitrary attributes and regex support · Issue #1515 · IQSS/dataverse - https://github.com/IQSS/dataverse/issues/1515
15:34 pdurbin bencomp: send more developers! :)
15:35 bencomp let me get back to you about that
15:47 pdurbin ok :)
15:54 axfelix joined #dataverse
15:55 pdurbin msv is a very handy tool! add MSV (Multi Schema Validator) to tools, add ToC #2579 · IQSS/dataverse@6a9de9c - https://github.com/IQSS/dataverse/commit/6a9de9c
15:56 pdurbin for validating XML against a schema
15:59 dataverse-user joined #dataverse
16:24 mjturk joined #dataverse
16:54 metamattj joined #dataverse
18:20 garnett joined #dataverse
18:26 garnett joined #dataverse
18:56 garnett joined #dataverse
19:17 pdurbin sbmarks: do you plan to use harvesting? (OAI-PMH)
19:20 bencomp joined #dataverse
20:03 cnk joined #dataverse
20:39 SlavaT joined #dataverse
20:42 SlavaT hi there! is it possible to get somewhere documentation how to use https://github.com/IQSS/dataverse-client-python ? I need to know about add dataverse/dataset possibilities of this cliet.
20:43 pdurbin SlavaT: hi! I see a test here for creating a dataverse: https://github.com/IQSS/dataverse-client-python/blob/d6e2199f21cddffac37d94d4e39caeaece70ee7e/dataverse/test/test_dataverse.py#L132
20:44 pdurbin and a test for creating a dataset: https://github.com/IQSS/dataverse-client-python/blob/d6e2199f21cddffac37d94d4e39caeaece70ee7e/dataverse/test/test_dataverse.py#L279
20:44 pdurbin SlavaT: does that help?
20:45 pdurbin SlavaT: also, I'm happy to help you with the Search API if you're still having trouble. I saw your message at http://irclog.iq.harvard.edu/dataverse/2015-09-09#i_23775
20:45 SlavaT yes, thanks! We're working on framework to analyze and visualize our datasets and this functionality is really critical.
20:47 SlavaT Still a lot of things to do but it's fully integrated with Dataverse already.
20:47 pdurbin nice!
20:47 SlavaT well, just for test: http://gis.sandbox.socialhistoryservices.org/collabs/dashboard?action=visualize&yearmin=1960&yearmax=2013&loc=36&loc=554&loc=398&loc=804&loc=124&loc=528&dataset=hdl%3A10622%2F4X6NCK%3A114%3A115
20:49 pdurbin SlavaT: looks great!
20:50 SlavaT we want to combine and analyze all datasets in one dataframe. regression analysis, correlations, etc
20:51 pdurbin nice
20:51 pdurbin SlavaT: hey, this is related: https://github.com/IQSS/data-viz
20:52 SlavaT oh yeah, I already thought how to connect it to TwoRavens. :)
20:53 SlavaT our framework is also python Flask https://github.com/IISH/dpe
20:54 pdurbin nice. this looks like a new repo
20:56 SlavaT oh yeah, fresh one. :) I've started to improve data processing engine recently.
20:58 pdurbin SlavaT: hey, how interested are you in DDI? that's what I'm working on now: https://github.com/IQSS/dataverse/issues/2579
20:59 SlavaT Yes, DDI is something we also need. Nice one!
21:00 pdurbin ok. cool. if you have any DDI files you'd like me to test with, please let me know
21:01 SlavaT thanks, I'll ask our researchers if they already have something to test already.
21:02 pdurbin awesome. something that could be public (checked into the codebase as a test) would be ideal
21:04 SlavaT yes, many thanks!
21:06 pdurbin oh sure. oh, by the way, romainM was recently poking at the API too. see http://irclog.iq.harvard.edu/dataverse/2015-09-23
21:06 pdurbin and he's using Python, I think
21:10 garnett joined #dataverse
21:18 garnett joined #dataverse
22:01 axfelix joined #dataverse
22:44 bencomp First PR against the new 4.3 branch, isn't it? https://github.com/IQSS/dataverse/pull/2589
22:55 pdurbin bencomp: probably. do you have tests for all these changes? :)
22:58 bencomp pdurbin: nope. tests are going in a different branch
23:00 pdurbin ok
23:01 pdurbin bencomp: you could be working on shib regex groups :)
23:01 bencomp small, basic tests that is
23:02 pdurbin or fixing bugs :)
23:02 bencomp would you pay me? DANS doesn't pay me to program
23:03 pdurbin me personally? :)
23:06 bencomp would you? :)
23:07 * pdurbin thinks
23:07 pdurbin bencomp: I do appreciate what you're doing :)
23:08 bencomp pdurbin: that helps :)
23:10 pdurbin :)
23:10 pdurbin I value it, even! not sure how much out of my own pocket though ;)
23:28 bencomp https://github.com/IQSS/dataverse/blob/4.3/src/main/java/edu/harvard/iq/dataverse/DatasetVersionUI.java#L375 am I missing something, or does this line do the same as 383?
23:36 pdurbin bencomp: looks like a copy/paste
23:38 bencomp I just recently learned the meaning of DRY
23:39 bencomp but I'm guessing sorting a list twice isn't really efficient :P
23:40 pdurbin :)
23:40 pdurbin bencomp: have you read The Pragmatic Programmer? great book
23:40 bencomp I saw DANS has it
23:41 bencomp but no, I haven't read it
23:41 pdurbin https://pragprog.com/book/tpp/the-pragmatic-programmer
23:41 pdurbin well worth reading
23:42 pdurbin bencomp: here, just read this for now: http://blog.codinghorror.com/a-pragmatic-quick-reference/
23:42 pdurbin bencomp: you already Don't Live with Broken Windows :)
23:43 bencomp I should sleep now
23:44 pdurbin I should go home. My youngest just lost her first tooth.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.