IQSS logo

IRC log for #dataverse, 2018-12-18

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
07:55 poikilotherm joined #dataverse
08:04 jri joined #dataverse
09:50 MrK joined #dataverse
10:01 MrK Hi, do you guys have a problem with publishing test datasets? I receive "Response code: 422, found unpermitted parameter: :validate" for some reason.
10:02 poikilotherm Hi Mrk :-) Sry, can't help with this... Maybe pdurbin is around in an hour or so.
10:03 poikilotherm Any progress on the Flyway stuff? I am eager to try this and base my stuff on it... ;-)
10:05 MrK I had to do some funcionalities so I had no time to look at it :<, do you remember if it was aproved to do it how we discussed?
10:06 poikilotherm @landreev worries about the extra work for past versions, but likes the flyway stuff overall
10:07 poikilotherm Maybe add the "past version support" if it is simple to do and otherwise stick with "supported from version X"?
10:34 MrK Ah, atm I have to do some functionalities, you can play around with flyway if you want this sooner :P
11:08 poikilotherm Alright, I see if I find some time :-)
11:50 pdurbin MrK: I was having trouble publishing on my laptop with test DataCite credentials but I didn't get that error, it was just hanging forever: https://github.com/IQSS/dataverse/issues/5393#issuecomment-447111860
12:56 juancorr joined #dataverse
13:25 pdurbin juancorr: thanks for replying at https://groups.google.com/d/msg/dataverse-community/b29o4ri2m9s/K18xPHiUBgAJ !
13:31 poikilotherm Mornin' pdurbin
13:31 poikilotherm Is Harvard dataverse OAI harvestable from outside?
13:31 poikilotherm I would like to test the harvest client code I just wrote
13:33 pdurbin poikilotherm: should be. Have you seen "List of Dataverse installation OAI-PMH (Harvesting) URLs and sets"? Here: https://docs.google.com/spreadsheets/d/12cxymvXCqP_kCsLKXQD32go79HBWZ1vU_tdG4kvP5S8/edit?usp=sharing
13:33 poikilotherm Nope.
13:34 poikilotherm Looked in the docs, but may have overlooked it
13:34 poikilotherm The docs could be more specific about the endpoint URL... ;-)
13:34 pdurbin The "About" tab links to the original conversation on the google group.
13:34 poikilotherm Yeah, I know, open issue and do PR
13:34 pdurbin poikilotherm: well, there is an open issue. I'd love for you to make pull request. One sec.
13:35 pdurbin https://github.com/IQSS/dataverse/issues/4203
13:35 pdurbin please please make a pull request
13:35 poikilotherm LOL
13:38 pdurbin with sugar on top
13:39 poikilotherm Do you think it would be alright to include this in my current PR? I already changed docs about metadata exporting
13:39 poikilotherm Otherwise I need to rebase ...
13:40 poikilotherm And it would take another roundtrip from kcondon for a rather simple doc change
13:41 pdurbin sure, that's fine. please help remind me to move it to QA at the appropriate time.
13:42 poikilotherm Right
13:42 poikilotherm This still needs code review once I pushed todays changes
13:42 poikilotherm Removed the old timing stuff completely now
13:45 poikilotherm I'll write some docs now while waiting for the timer... ;-)
13:46 pdurbin You have to wait until 2am? ;)
13:46 poikilotherm Nope, 3pm my timezone ;-)
13:46 poikilotherm That's 9am your timezone
13:46 pdurbin ok, could be worse
13:46 poikilotherm It's about the harvesting timers, those are configurable
13:47 poikilotherm For the export timer I think about clock-faking ;-)
13:47 pdurbin Oh, I guess you could link to that spreadsheet while you're in there.
13:49 poikilotherm That was the plan... ;-)
13:49 pdurbin longer term that information should be in https://services.dataverse.harvard.edu/miniverse/map/installations-json
13:50 poikilotherm Sound reasonable. What stops you from adding those bits?
13:51 poikilotherm Maybe add a contact, too
13:51 pdurbin it's a long story
13:51 juancorr pdurbin: You are welcome
13:52 pdurbin juancorr: hey, do you know approximately what year and month you went into production?
13:55 pdurbin poikilotherm: here's part of the story, if you're interested: https://github.com/IQSS/miniverse/issues/61
13:57 juancorr Yes I do. e-cienciaDatos was into production in January 2017
13:58 poikilotherm pdurbin: I see.
13:58 poikilotherm Maybe switch to sth. else if this is getting you in maintenance hell... ;-)
14:01 pdurbin juancorr: thanks! I updated our spreadsheet. (Different spreadsheet than poikilotherm and I are talking about.) :)
14:03 pdurbin poikilotherm: there's also a list of installations at https://github.com/IQSS/metrics.dataverse.org/blob/v0.1.0/config.json.sample
14:03 pdurbin poikilotherm: we've got to get you in production so you can join the party :)
14:06 poikilotherm [2018-12-18T14:00:00.021+0000] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.harve‚Äčst.client.HarvesterServiceBean] [tid: _ThreadID=149 _ThreadName=__ejb-thread-pool13] [timeMillis: 1545141600021] [levelValue: 800] [[
14:06 poikilotherm Running harvesting client: id=1 name=Test using admin user id=1 name=Dataverse Admin]]
14:07 pdurbin Does that mean it worked?
14:07 poikilotherm Yeah
14:07 pdurbin nice, good job
14:07 poikilotherm Do you think harvesting will be under an hour?
14:08 poikilotherm The current approach was let harvesters run on independent timers, so they are decoupled
14:09 pdurbin You're harvesting from Harvard Dataverse? I have no idea. Sorry.
14:09 poikilotherm Now I just look every hour if a client needs to do harvesting. And if this takes longer than one hour, this might cause issues
14:09 poikilotherm Left a note about this in the docs
14:10 pdurbin great
14:10 poikilotherm Er... the java docs ;-)
14:10 poikilotherm Maybe I should add a hint in the docs, too
14:10 pdurbin up to you
14:16 poikilotherm Done ;-)
14:16 poikilotherm I added timers to dev docs anyway, so left a note there
14:17 pdurbin much appreciated
14:21 pdurbin poikilotherm: speaking of harvesting, do you have any feedback on https://github.com/IQSS/dataverse/issues/5402 which was opened by jonas42?
14:21 poikilotherm Yeah. Code in HarvestServiceBean needs refactoring for this.
14:22 poikilotherm To be precise: the hole "client" side stuff should be refactored for this.
14:22 poikilotherm IMHO the code "as is" is not ready for such an extension
14:23 poikilotherm Should I add a note on the GH issue about this?
14:25 poikilotherm Should I add a note on the GH issue about this?
14:25 poikilotherm Ups
14:25 poikilotherm Sry
14:25 poikilotherm Wrong windows
14:44 poikilotherm pdurbin: to what federation is Harvard Dataverse attached to?
14:45 poikilotherm Could be interesting to add some services at FZJ to this...
14:45 poikilotherm It seems like it is not eduGain
14:45 poikilotherm (We are not present in the list)
14:48 poikilotherm Is InCommon correct?
14:50 poikilotherm Hmm doesn't look correct - at Dataverse I could login via RWTH Aachen, but RWTH is not listed at InCommon
15:03 pdurbin poikilotherm: yes, InCommon
15:04 poikilotherm Ok. InCommon is part of eduGain.
15:04 pdurbin You should be able to log in via RWTH Aachen at https://dataverse.unc.edu too.
15:04 poikilotherm Could you do a quick test? Can you login via Shib at https://jugit.fz-juelich.de/?
15:05 poikilotherm RWTH seems to have joind InCommon then, because FZJ is not present in the list...
15:06 pdurbin poikilotherm: "Sign-in failed because Email can't be blank and Notification email can't be blank." https://i.imgur.com/v3DYdA0.png
15:06 poikilotherm Nive...
15:06 poikilotherm -v +c
15:07 poikilotherm Harvard IdP seems not to send the email attrib
15:07 pdurbin "Sign-in using shibboleth auth failed" etc
15:07 pdurbin probably a privacy thing
15:07 poikilotherm Great...
15:08 poikilotherm That renders collaboration pretty useless
15:08 pdurbin If that app joined R&S it might work. But that's an InCommon thing. Research & Scholarship.
15:09 poikilotherm joined #dataverse
15:14 pdurbin I prefer OAuth where users are in control over the decision to release information that the app is asking for (such as an email address). With SAML/Shibboleth the user has no control.
15:17 poikilotherm YEah.
15:18 poikilotherm But I don't know of any federation as with eduGain
15:19 pdurbin Here's where I talk about R&S: http://guides.dataverse.org/en/4.9.4/installation/shibboleth.html#identity-federation
15:47 pdurbin pameyer: mornin
15:57 pameyer pdurbin: morning - left things connected when I left last night
15:58 pameyer and cbc7dd62a90a4fb796e942f26e8037aee6b5cf74 on 5400-revert-f-view does pass all the ITs
15:58 pameyer develop? let's find out
15:58 pdurbin thank you!
15:59 pdurbin As I was saying in person, publishing is hanging for me on phoenix and my laptop when using DataCite. Steve is on EZID and isn't having any trouble.
16:34 pameyer pdurbin: works for me; I get the one expected/known failure
16:45 pdurbin pameyer: phew. Good. I just opened https://github.com/IQSS/dataverse/issues/5409
17:00 pdurbin Everyone is welcome to join the community call, which is just starting: https://dataverse.org/community-calls
17:01 pdurbin andrewSC bjonnh bricas candy` dzho icarito[m] jonas42 JonathanNeal jri sivoais xarthisius yoh ^^
17:01 pdurbin and toward the end of the call we plan to talk about Docker and Kubernetes
18:18 jri joined #dataverse
19:02 jri joined #dataverse
19:49 jri joined #dataverse
19:52 poikilotherm joined #dataverse
19:53 poikilotherm Good evening :-)
19:53 poikilotherm pdurbin I wanted to thank you for taking the notes for the Dataverse on Docker & Kubernetes stuff :-)
19:55 poikilotherm Very much appreciated as I then could concentrate on searching vocabulary ;-)
20:01 pameyer poikilotherm thanks for driving that part of the conversation
20:02 poikilotherm Sure. Altough it lead to no real insights :-/
20:08 pameyer I got the impression that there were multiple sources mentioning similar things - but don't know if that counts as insight or not
20:10 jri joined #dataverse
20:13 poikilotherm https://www.oreilly.com/library/view/refactoring-improving-the/9780134757681/ A new revision of the refactoring bible has been released :-)
20:14 poikilotherm pameyer: yeah, it seemed like there is interest and people are likely-minded towards my proposals. But there were not many people around and Slava, the only guy in town very near to running Dataverse on Kubernetes in production, left early
20:14 poikilotherm Dunno if that is a good sign or not
20:14 poikilotherm Hope I didn't make him upset :-/
20:20 pdurbin poikilotherm: sure, I was happy to take notes. My usual pattern is to copy and past a plain text version of notes into a follow up email on the Google Group. I'll do this for the normal call. Do you want to do this for the Docker call?
20:21 poikilotherm Hmm maybe we should just send a follow up and provide the link to the doc?
20:21 poikilotherm This could be tweeted and posted in the GH issue, too
20:28 pdurbin Up to you. I'll do my normal thing for the regular meeting notes.
20:28 poikilotherm :-)
20:31 jri joined #dataverse
20:51 jri joined #dataverse
21:08 pdurbin poikilotherm: thanks for organizing that meeting. I hope you think it went ok.
21:11 poikilotherm :-)
21:12 poikilotherm pdurbin: you should try to get on this list: https://www.openarchives.org/pmh/tools/
21:12 jri joined #dataverse
21:12 poikilotherm (Dataverse on this list)
21:14 pdurbin interesting, not a bad idea
21:17 poikilotherm All the other repos are already present... ;-)
21:21 pdurbin Would each installation add themselves? Scholars Portal? UNC Dataverse?
21:26 poikilotherm Nope, I understood the list that projects/tools are listed there.
21:26 poikilotherm See DSPace, Fedora, ... on the list
21:27 poikilotherm pdurbin: do you think #4203 ist only about adding docs or should the UI be changed, too, to display the URL?
21:27 poikilotherm And should there be documentation just about the /oai endpoint or also about URLs to OAI sets?
21:36 poikilotherm Maybe you can take a look at https://github.com/poikilotherm/dataverse/commit/0258d190218d8212183f6123445f9bd1c65a03f9
21:39 poikilotherm That's what I did so far for #4203
21:40 pdurbin I think you should stop at the doc change for now. Thanks for working on this!
21:40 poikilotherm Do you think this is sufficient and really solves the issue?
21:40 poikilotherm Any other places where this should be referenced?
21:42 poikilotherm Maybe at the harvesting client doc?
21:42 pdurbin Well, "definition of done" is something we talk about all the time. You could leave a comment and ask Thalia what she thinks about the doc improvement you just made. She used to hang out in IRC but I haven't seen her in a bit.
21:45 poikilotherm Done
22:20 poikilotherm Alright guys, I'm off for today :-) Read you tomorrow

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.