Time
S
Nick
Message
07:55 poikilotherm joined #dataverse
08:04 jri joined #dataverse
09:50 MrK joined #dataverse
10:01 MrK
Hi, do you guys have a problem with publishing test datasets? I receive "Response code: 422, found unpermitted parameter: :validate" for some reason.
10:02 poikilotherm
Hi Mrk :-) Sry, can't help with this... Maybe pdurbin is around in an hour or so.
10:03 poikilotherm
Any progress on the Flyway stuff? I am eager to try this and base my stuff on it... ;-)
10:05 MrK
I had to do some funcionalities so I had no time to look at it :<, do you remember if it was aproved to do it how we discussed?
10:06 poikilotherm
@landreev worries about the extra work for past versions, but likes the flyway stuff overall
10:07 poikilotherm
Maybe add the "past version support" if it is simple to do and otherwise stick with "supported from version X"?
10:34 MrK
Ah, atm I have to do some functionalities, you can play around with flyway if you want this sooner :P
11:08 poikilotherm
Alright, I see if I find some time :-)
11:50 pdurbin
MrK: I was having trouble publishing on my laptop with test DataCite credentials but I didn't get that error, it was just hanging forever: https://github.com/IQSS/dataverse/issues/5393#issuecomment-447111860
12:56 juancorr joined #dataverse
13:25 pdurbin
juancorr: thanks for replying at https://groups.google.com/d/msg/dataverse-community/b29o4ri2m9s/K18xPHiUBgAJ !
13:31 poikilotherm
Mornin' pdurbin
13:31 poikilotherm
Is Harvard dataverse OAI harvestable from outside?
13:31 poikilotherm
I would like to test the harvest client code I just wrote
13:33 pdurbin
poikilotherm: should be. Have you seen "List of Dataverse installation OAI-PMH (Harvesting) URLs and sets"? Here: https://docs.google.com/spreadsheets/d/12cxymvXCqP_kCsLKXQD32go79HBWZ1vU_tdG4kvP5S8/edit?usp=sharing
13:33 poikilotherm
Nope.
13:34 poikilotherm
Looked in the docs, but may have overlooked it
13:34 poikilotherm
The docs could be more specific about the endpoint URL ... ;-)
13:34 pdurbin
The "About" tab links to the original conversation on the google group.
13:34 poikilotherm
Yeah, I know, open issue and do PR
13:34 pdurbin
poikilotherm: well, there is an open issue. I'd love for you to make pull request. One sec.
13:35 pdurbin
https://github.com/IQSS/dataverse/issues/4203
13:35 pdurbin
please please make a pull request
13:35 poikilotherm
LOL
13:38 pdurbin
with sugar on top
13:39 poikilotherm
Do you think it would be alright to include this in my current PR? I already changed docs about metadata exporting
13:39 poikilotherm
Otherwise I need to rebase ...
13:40 poikilotherm
And it would take another roundtrip from kcondon for a rather simple doc change
13:41 pdurbin
sure, that's fine. please help remind me to move it to QA at the appropriate time.
13:42 poikilotherm
Right
13:42 poikilotherm
This still needs code review once I pushed todays changes
13:42 poikilotherm
Removed the old timing stuff completely now
13:45 poikilotherm
I'll write some docs now while waiting for the timer... ;-)
13:46 pdurbin
You have to wait until 2am? ;)
13:46 poikilotherm
Nope, 3pm my timezone ;-)
13:46 poikilotherm
That's 9am your timezone
13:46 pdurbin
ok, could be worse
13:46 poikilotherm
It's about the harvesting timers, those are configurable
13:47 poikilotherm
For the export timer I think about clock-faking ;-)
13:47 pdurbin
Oh, I guess you could link to that spreadsheet while you're in there.
13:49 poikilotherm
That was the plan... ;-)
13:49 pdurbin
longer term that information should be in https://services.dataverse.harvard.edu/miniverse/map/installations-json
13:50 poikilotherm
Sound reasonable. What stops you from adding those bits?
13:51 poikilotherm
Maybe add a contact, too
13:51 pdurbin
it's a long story
13:51 juancorr
pdurbin: You are welcome
13:52 pdurbin
juancorr: hey, do you know approximately what year and month you went into production?
13:55 pdurbin
poikilotherm: here's part of the story, if you're interested: https://github.com/IQSS/miniverse/issues/61
13:57 juancorr
Yes I do. e-cienciaDatos was into production in January 2017
13:58 poikilotherm
pdurbin: I see.
13:58 poikilotherm
Maybe switch to sth. else if this is getting you in maintenance hell... ;-)
14:01 pdurbin
juancorr: thanks! I updated our spreadsheet. (Different spreadsheet than poikilotherm and I are talking about.) :)
14:03 pdurbin
poikilotherm: there's also a list of installations at https://github.com/IQSS/metrics.dataverse.org/blob/v0.1.0/config.json.sample
14:03 pdurbin
poikilotherm: we've got to get you in production so you can join the party :)
14:06 poikilotherm
[2018-12-18T14:00:00.021+0000] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.harvest.client.HarvesterServiceBean] [tid: _ThreadID=149 _ThreadName=__ejb-thread-pool13] [timeMillis: 1545141600021] [levelValue: 800] [[
14:06 poikilotherm
Running harvesting client: id=1 name=Test using admin user id=1 name=Dataverse Admin]]
14:07 pdurbin
Does that mean it worked?
14:07 poikilotherm
Yeah
14:07 pdurbin
nice, good job
14:07 poikilotherm
Do you think harvesting will be under an hour?
14:08 poikilotherm
The current approach was let harvesters run on independent timers, so they are decoupled
14:09 pdurbin
You're harvesting from Harvard Dataverse? I have no idea. Sorry.
14:09 poikilotherm
Now I just look every hour if a client needs to do harvesting. And if this takes longer than one hour, this might cause issues
14:09 poikilotherm
Left a note about this in the docs
14:10 pdurbin
great
14:10 poikilotherm
Er... the java docs ;-)
14:10 poikilotherm
Maybe I should add a hint in the docs, too
14:10 pdurbin
up to you
14:16 poikilotherm
Done ;-)
14:16 poikilotherm
I added timers to dev docs anyway, so left a note there
14:17 pdurbin
much appreciated
14:21 pdurbin
poikilotherm: speaking of harvesting, do you have any feedback on https://github.com/IQSS/dataverse/issues/5402 which was opened by jonas42?
14:21 poikilotherm
Yeah. Code in HarvestServiceBean needs refactoring for this.
14:22 poikilotherm
To be precise: the hole "client" side stuff should be refactored for this.
14:22 poikilotherm
IMHO the code "as is" is not ready for such an extension
14:23 poikilotherm
Should I add a note on the GH issue about this?
14:25 poikilotherm
Should I add a note on the GH issue about this?
14:25 poikilotherm
Ups
14:25 poikilotherm
Sry
14:25 poikilotherm
Wrong windows
14:44 poikilotherm
pdurbin: to what federation is Harvard Dataverse attached to?
14:45 poikilotherm
Could be interesting to add some services at FZJ to this...
14:45 poikilotherm
It seems like it is not eduGain
14:45 poikilotherm
(We are not present in the list)
14:48 poikilotherm
Is InCommon correct?
14:50 poikilotherm
Hmm doesn't look correct - at Dataverse I could login via RWTH Aachen, but RWTH is not listed at InCommon
15:03 pdurbin
poikilotherm: yes, InCommon
15:04 poikilotherm
Ok. InCommon is part of eduGain.
15:04 pdurbin
You should be able to log in via RWTH Aachen at https://dataverse.unc.edu too.
15:04 poikilotherm
Could you do a quick test? Can you login via Shib at https://jugit.fz-juelich.de/?
15:05 poikilotherm
RWTH seems to have joind InCommon then, because FZJ is not present in the list...
15:06 pdurbin
poikilotherm: "Sign-in failed because Email can't be blank and Notification email can't be blank." https://i.imgur.com/v3DYdA0.png
15:06 poikilotherm
Nive...
15:06 poikilotherm
-v +c
15:07 poikilotherm
Harvard IdP seems not to send the email attrib
15:07 pdurbin
"Sign-in using shibboleth auth failed" etc
15:07 pdurbin
probably a privacy thing
15:07 poikilotherm
Great...
15:08 poikilotherm
That renders collaboration pretty useless
15:08 pdurbin
If that app joined R&S it might work. But that's an InCommon thing. Research & Scholarship.
15:09 poikilotherm joined #dataverse
15:14 pdurbin
I prefer OAuth where users are in control over the decision to release information that the app is asking for (such as an email address). With SAML/Shibboleth the user has no control.
15:17 poikilotherm
YEah.
15:18 poikilotherm
But I don't know of any federation as with eduGain
15:19 pdurbin
Here's where I talk about R&S: http://guides.dataverse.org/en/4.9.4/installation/shibboleth.html#identity-federation
15:47 pdurbin
pameyer: mornin
15:57 pameyer
pdurbin: morning - left things connected when I left last night
15:58 pameyer
and cbc7dd62a90a4fb796e942f26e8037aee6b5cf74 on 5400-revert-f-view does pass all the ITs
15:58 pameyer
develop? let's find out
15:58 pdurbin
thank you!
15:59 pdurbin
As I was saying in person, publishing is hanging for me on phoenix and my laptop when using DataCite. Steve is on EZID and isn't having any trouble.
16:34 pameyer
pdurbin: works for me; I get the one expected/known failure
16:45 pdurbin
pameyer: phew. Good. I just opened https://github.com/IQSS/dataverse/issues/5409
17:00 pdurbin
Everyone is welcome to join the community call, which is just starting: https://dataverse.org/community-calls
17:01 pdurbin
andrewSC bjonnh bricas candy` dzho icarito[m] jonas42 JonathanNeal jri sivoais xarthisius yoh ^^
17:01 pdurbin
and toward the end of the call we plan to talk about Docker and Kubernetes
18:18 jri joined #dataverse
19:02 jri joined #dataverse
19:49 jri joined #dataverse
19:52 poikilotherm joined #dataverse
19:53 poikilotherm
Good evening :-)
19:53 poikilotherm
pdurbin I wanted to thank you for taking the notes for the Dataverse on Docker & Kubernetes stuff :-)
19:55 poikilotherm
Very much appreciated as I then could concentrate on searching vocabulary ;-)
20:01 pameyer
poikilotherm thanks for driving that part of the conversation
20:02 poikilotherm
Sure. Altough it lead to no real insights :-/
20:08 pameyer
I got the impression that there were multiple sources mentioning similar things - but don't know if that counts as insight or not
20:10 jri joined #dataverse
20:13 poikilotherm
https://www.oreilly.com/library/view/refactoring-improving-the/9780134757681/ A new revision of the refactoring bible has been released :-)
20:14 poikilotherm
pameyer: yeah, it seemed like there is interest and people are likely-minded towards my proposals. But there were not many people around and Slava, the only guy in town very near to running Dataverse on Kubernetes in production, left early
20:14 poikilotherm
Dunno if that is a good sign or not
20:14 poikilotherm
Hope I didn't make him upset :-/
20:20 pdurbin
poikilotherm: sure, I was happy to take notes. My usual pattern is to copy and past a plain text version of notes into a follow up email on the Google Group. I'll do this for the normal call. Do you want to do this for the Docker call?
20:21 poikilotherm
Hmm maybe we should just send a follow up and provide the link to the doc?
20:21 poikilotherm
This could be tweeted and posted in the GH issue, too
20:28 pdurbin
Up to you. I'll do my normal thing for the regular meeting notes.
20:28 poikilotherm
:-)
20:31 jri joined #dataverse
20:51 jri joined #dataverse
21:08 pdurbin
poikilotherm: thanks for organizing that meeting. I hope you think it went ok.
21:11 poikilotherm
:-)
21:12 poikilotherm
pdurbin: you should try to get on this list: https://www.openarchives.org/pmh/tools/
21:12 jri joined #dataverse
21:12 poikilotherm
(Dataverse on this list)
21:14 pdurbin
interesting, not a bad idea
21:17 poikilotherm
All the other repos are already present... ;-)
21:21 pdurbin
Would each installation add themselves? Scholars Portal? UNC Dataverse?
21:26 poikilotherm
Nope, I understood the list that projects/tools are listed there.
21:26 poikilotherm
See DSPace, Fedora, ... on the list
21:27 poikilotherm
pdurbin: do you think #4203 ist only about adding docs or should the UI be changed, too, to display the URL ?
21:27 poikilotherm
And should there be documentation just about the /oai endpoint or also about URLs to OAI sets?
21:36 poikilotherm
Maybe you can take a look at https://github.com/poikilotherm/dataverse/commit/0258d190218d8212183f6123445f9bd1c65a03f9
21:39 poikilotherm
That's what I did so far for #4203
21:40 pdurbin
I think you should stop at the doc change for now. Thanks for working on this!
21:40 poikilotherm
Do you think this is sufficient and really solves the issue?
21:40 poikilotherm
Any other places where this should be referenced?
21:42 poikilotherm
Maybe at the harvesting client doc?
21:42 pdurbin
Well, "definition of done" is something we talk about all the time. You could leave a comment and ask Thalia what she thinks about the doc improvement you just made. She used to hang out in IRC but I haven't seen her in a bit.
21:45 poikilotherm
Done
22:20 poikilotherm
Alright guys, I'm off for today :-) Read you tomorrow
