Time
S
Nick
Message
08:19 jri joined #dataverse
09:15 pdurbin
poikilotherm: I linked to the 5 minute video about CC0 vs CC-BY that I thought you might be interested in: https://github.com/IQSS/dataverse/issues/1753#issuecomment-591753956
09:39 stefankasberger joined #dataverse
10:09 poikilotherm
Good morning pdurbin
10:10 poikilotherm
Early bird again :-D
10:10 poikilotherm
Yeah, saw that but had no chance to view yet ;-)
10:10 poikilotherm
Thanks for linking! :-)
10:16 poikilotherm
pdurbin: thx for the heads up in https://github.com/IQSS/dataverse/issues/6561
10:17 poikilotherm
I have sth in the back of my head that I asked about the very same thing regarding displayOnCreate + optional a while ago. Wasn't there an issue for that already?
10:42 pdurbin
not that I recall
10:42 pdurbin
poikilotherm: oh, I blogged about the OIDC feature you added: http://blog.greptilian.com/2020/02/25/research-data-repository-software-comparison/
10:47 skasberger joined #dataverse
10:48 pdurbin
thanks again :)
10:55 poikilotherm
Outstanding feature?
10:58 poikilotherm
Ah OK read your post
10:59 pdurbin
:)
10:59 poikilotherm
Thx :-)
10:59 pdurbin
sure, I think it's next generation feature
10:59 poikilotherm
It's a "next-gen repo feature" :-D
10:59 pdurbin
jinx
10:59 poikilotherm
Hihihi
10:59 poikilotherm
Yeah
11:00 pdurbin
Are you going to implement the rest of them for us? :)
11:00 poikilotherm
We also support CC licenses
11:00 poikilotherm
LOL
11:00 poikilotherm
And sitemaps should have a yes, too
11:00 pdurbin
Hmm, do you mind leaving a comment about sitemaps?
11:01 poikilotherm
Done
11:01 pdurbin
Thanks! I'm trying to only leave one comment at a time and my current open comment is about a 2.5 GB limit.
11:01 pdurbin
Which you are welcome to read and further comment on, of course.
11:02 poikilotherm
Oh and don't we have COUNTER support?
11:02 poikilotherm
https://www.projectcounter.org/
11:03 pdurbin
Well, Make Data Count is based on COUNTER concepts.
11:11 poikilotherm
I dug in the logs... We talked about this very issue on 2019-05-10 here http://irclog.iq.harvard.edu/dataverse/2019-05-10#i_93140
11:12 poikilotherm
I said I'd create an issue, but it looks like I never did. Forgot about it over drilling holes in concrete
11:12 pdurbin
:)
11:17 poikilotherm
We talked about tweaking citation.tsv today, too
11:17 poikilotherm
Like in https://github.com/IQSS/dataverse/issues/6561
11:18 poikilotherm
And I was talking to Doro this morning about more metadata standards and their representation in Dataverse
11:18 poikilotherm
I wonder if might be a good idea to move those TSVs to a separated repo
11:18 poikilotherm
That everyone could fork
11:18 poikilotherm
And where you can apply your changes
11:19 poikilotherm
And use git to keep track of upstream changes
11:19 poikilotherm
To give an example: my colleagues would be happy if we could remove the CV for author ids and only have ORCID
11:20 poikilotherm
But I don't want to fork the whole applicatio
11:20 poikilotherm
And it would be very nice to keep track of things more easily
11:22 pdurbin
If you want this, please create an issue for it.
11:22 poikilotherm
Via GH fork we could have at least track some people and what they are doing about metadata blocks in their installations
11:23 poikilotherm
Do think this is a good idea?
11:23 poikilotherm
+you
11:24 poikilotherm
Do you think there is a chance that important people at IQSS might see value in this?
11:24 pdurbin
I'm thinking about something else... how I really want lauch dates for all installations of Dataverse so we can make an accurate growth of adoption over time: https://github.com/IQSS/dataverse-installations/issues/7#issuecomment-591919312
11:25 pdurbin
skasberger: when did you launch? ^^
11:25 poikilotherm
I just read your post :-D
11:25 poikilotherm
Isn't his nick stefankasberger ?
11:25 pdurbin
Do I have the wrong cheese man?!?!
11:26 poikilotherm
Seems like two accounts for the same cheese :-D
12:01 poikilotherm
Ha epic issue number for my crazy idea https://github.com/IQSS/dataverse/issues/6700
12:18 skasberger
have not launched so far. still having trouble with shibd process. it is not starting properly. the socket is not created, and the RAM is filled up continuisly.
12:18 skasberger
have not launched so far. still having trouble with shibd process. it is not starting properly. the socket is not created, and the RAM is filled up continuoisly.
12:18 pdurbin
skasberger: sorry, I mean your original launch date
12:18 pdurbin
probably 2019 or 2018
12:18 pdurbin
poikilotherm: good stuff. I just left this comment: https://github.com/IQSS/dataverse/issues/4451#issuecomment-591941172
12:18 poikilotherm
skasberger: what federation did you join?
12:18 skasberger
Error: Internal Error - Failed to download metadata from /Shibboleth.sso/DiscoFeed.
12:19 skasberger
edugain
12:19 poikilotherm
Pretty please try with a smaller fed first
12:19 poikilotherm
As I wrote yesterday: shibboleth as an SP sucks when it comes to those large feds
12:19 poikilotherm
The lists of IdPs takes forever to load, hours are normal
12:20 skasberger
do not really know. I guess 2017 or 2018.
12:20 poikilotherm
Here in Germany there is the DFN AAI. I don't know if there is sth similiar in Austria
12:21 poikilotherm
They have a test federation, so one can check quickly if the setup itself is working OK
12:28 pdurbin
skasberger: did you have Shibboleth working on your old installation?
12:38 pdurbin
Or is this a new feature for you?
12:42 skasberger
no, used it before. its still the same server. but since some upgrades (or maybe a restart) the shibd does not work as expected.
12:44 pdurbin
Hmm, we did change something but it was quite a while ago.
12:44 * pdurbin
thinks
12:45 skasberger
we dont believe it is an issue coming from dataverse.
12:45 pdurbin
phew
12:45 pdurbin
Where are you seeing this error? In a browser? In server.log? Error: Internal Error - Failed to download metadata from /Shibboleth.sso/DiscoFeed.
12:46 skasberger
this was on dataverse when loading shibboleth login page.
12:46 skasberger
in a prompt.
12:47 pdurbin
Does the equivalent of https://demo.dataverse.org/Shibboleth.sso/DiscoFeed work?
12:54 pdurbin
Going directly to that URL I mean.
13:04 pdurbin
This is a step I recommend at http://guides.dataverse.org/en/4.19/installation/shibboleth.html#verify-discofeed-and-metadata-urls
13:12 poikilotherm
skasberger: you might take a look at using tools like Keycloak to join the eduGain federation and replace Shibboleth
13:13 pdurbin
poikilotherm: upgrade from Shibboleth? :)
13:13 poikilotherm
Yeah. Becoming future proof.
13:13 poikilotherm
I'm hacking away on supporting more sophisticated stuff with OIDC for the future
13:14 pdurbin
Would this be supported? http://guides.dataverse.org/en/4.19/installation/shibboleth.html#identity-federation
13:14 poikilotherm
Like group syncs
13:14 poikilotherm
Definitely
13:14 poikilotherm
You can join Keycloak or other IDMs like Unity to any SAML federation
13:15 poikilotherm
pdurbin: test it! Head over to https://data-beta.fz-juelich.de/loginpage.xhtml?redirectPage=%2Fdataverse.xhtml
13:15 poikilotherm
Login with Harvard :-)
13:16 poikilotherm
HDF is using Unity IDM with edugain
13:20 pdurbin
poikilotherm: this is really cool. Have you considered adding a tooltip for eduPersonScopedAffiliation? I have no idea what to write.
13:21 poikilotherm
Well it all depends on your IdP sending the attributes... The IDM can only process what is being sent.
13:21 poikilotherm
HDF is pushing the eduPersonScopedAffiliation forward as an attribute
13:21 pdurbin
Sure, but I can talk to my IdP people. What would I tell them or ask them?
13:22 poikilotherm
And yes, I want to enable support for custom scopes and claims
13:22 pdurbin
"registration request submitted"
13:22 poikilotherm
You can check if its already pushed by your IdP
13:23 poikilotherm
Just login to the IDM and take a look at the attributes
13:24 poikilotherm
Should look like this https://i.imgur.com/4bPwlCO.png
13:24 poikilotherm
(Simply head to https://login.helmholtz-data-federation.de and login)
13:24 pdurbin
I think you have to approve my account first
13:26 poikilotherm
Oh? I have no idea... It's not me running the IDM. When coming from a german institution, you are "just accepted"
13:26 pdurbin
Oh. How long should I wait?
13:26 poikilotherm
Those rules are made by the IDM administrators
13:27 poikilotherm
Have you tried to login?
13:27 poikilotherm
There is an issue with the flow, it looks like you'll be stuck but you aren't
13:27 poikilotherm
And you should have received an email
13:28 poikilotherm
We are still squashing some UX bugs here with the folks running the show
13:29 pdurbin
I haven't checked my email yet but here's my user experience in the browser so far: https://github.com/IQSS/dataverse/issues/6701
13:29 Youssef_Ouahalou joined #dataverse
13:31 poikilotherm
Nice!
13:31 poikilotherm
I knew the new logo would be good :-D Now it's prominent on Github :-D
13:33 poikilotherm
My colleague told me that you just need to check your mail. You should have received an email that completes your signup
13:34 pdurbin
Yes. More screenshots: https://github.com/IQSS/dataverse/issues/6701#issuecomment-591972772
13:34 pdurbin
But now what?
13:34 poikilotherm
Yeah, now just login again
13:34 poikilotherm
That's the UX bug we need to squash
13:34 donsizemore joined #dataverse
13:35 pdurbin
Login again? Should I click "refresh" where in the window that still has "registration request submitted"?
13:35 poikilotherm
Just close that popup/overlay and click on the Harvard IdP again
13:35 poikilotherm
Or goto Dataverse and re-follow the Login buttins
13:38 poikilotherm
Hihihi @apw1388 is my collague running the IDM :-)
13:39 pdurbin
yet more screenshots: https://github.com/IQSS/dataverse/issues/6701#issuecomment-591975506
13:40 pdurbin
donsizemore: mornin'. Mr. Shib plays with OIDC and HarvardKey. ^^ :)
13:40 pdurbin
skasberger: this is looking promising!
13:41 poikilotherm
Beware: the Dataverse side is not perfectly ready for this yet. That's why I opened that huge bunch of issues
13:41 poikilotherm
I will need the new config JSON also for my ideas about group mapping and custom attributes
13:46 poikilotherm
pdurbin: did you click "Confirm"?
13:47 jri joined #dataverse
13:48 pdurbin
yes, just now, even more screenshots: https://github.com/IQSS/dataverse/issues/6701#issuecomment-591979742
13:49 poikilotherm
Maybe you ran into a timeout or the session expired due to time settings.
13:49 poikilotherm
Could you just re-initiate the login?
13:49 poikilotherm
Via the Login button?
13:49 poikilotherm
The state might have timed out already
13:50 poikilotherm
I'll check the logs in parallel
13:50 poikilotherm
Yeah, state timeout. [#|2020-02-27T13:47:28.513+0000|INFO|glassfish 4.1|edu.harvard.iq.dataverse.authorization.providers.oauth2.OAuth2LoginBackingBean|_ThreadID=32;_ThreadName=http-listener-1(5);_TimeMillis=1582811248513;_LevelValue=800;|
13:50 poikilotherm
State timeout|#]
13:52 poikilotherm
Here you go crazy man https://i.imgur.com/mNWAxDl.png
13:52 donsizemore
@poikilotherm he's not crazy man, he's Mr. Shib!
13:52 pdurbin
I'm Johnny Five!
13:52 poikilotherm
LOL
13:52 pdurbin
Screenshots of great success!! https://github.com/IQSS/dataverse/issues/6701#issuecomment-591981719
13:52 poikilotherm
iiiiiiiiiiiiiiiiiiiiiiinput!
13:53 pdurbin
still lumpy!
13:54 pdurbin
poikilotherm: does this mean that I can log into your installation of Dataverse with HarvardKey, ORCID, or GitHub and still be me?
13:54 poikilotherm
Yeah :-(
13:54 poikilotherm
But working on it! :-D
13:55 poikilotherm
To do that, you will need to login into the IDM first
13:55 poikilotherm
Then you need to connect that account to the other logins
13:55 poikilotherm
If you are feeling lucky, please try that
13:55 poikilotherm
Login to the IDM is at https://login.helmholtz-data-federation.de/
13:55 pdurbin
Can I get that IDM URL from your installation of Dataverse?
13:56 poikilotherm
On the profile page, there is a button "Associate another account"
13:56 poikilotherm
Nope. That would mean a UI change and we didn't do that :-D
13:57 poikilotherm
And this might be different for every kind of provider out there
13:57 poikilotherm
That's part of why I wanted to have an option in the config to show some description
13:57 poikilotherm
That could link to a helpfull page of explaining what happens, explain options like multi account login etc
13:59 donsizemore
@poikilotherm I am thrilled to see UNC on the list =)
14:00 donsizemore
(though I did not sully the database with an extraneous user account)
14:00 poikilotherm
It's just edugain
14:00 poikilotherm
If you feel like it, go ahead. It's just the test and training instance.
14:01 donsizemore
registration request submitted!
14:02 poikilotherm
:-D
14:04 pdurbin
poikilotherm: you should build custom guides and put your IDM link in them. The "your account has been created" email has a link to (custom or regular) guides. More screenshots and more on this: https://github.com/IQSS/dataverse/issues/6701#issuecomment-591986929
14:05 pdurbin
"... and explain in your custom guide that MULTIPLE LOGIN TO THE SAME ACCOUNT (for me HarvardKey, ORCID, GitHub) IS SUPPORTED! The dream of #3487 is alive!!! 🎉 🎉 🎉 I need to lie down. 🛏"
14:05 poikilotherm
Mr Shib goes crazy :-D
14:05 poikilotherm
I told you folks many times before this would be possible with OIDC+IDM
14:05 pdurbin
Yeah but I didn't believe you. :)
14:06 poikilotherm
LOL
14:07 pdurbin
What do I need to tell the Harvard IdP people to get this?
14:07 poikilotherm
We talked about a custom user guide this morning, as we want to give people a very brief intro, some term definitions etc
14:07 poikilotherm
Good to know this URL is also included in the signup URL
14:08 poikilotherm
Well they should think about getting rid of Shib
14:08 poikilotherm
At least as an SP
14:08 poikilotherm
But actually...
14:08 poikilotherm
I don't think you need to tell them anything
14:08 poikilotherm
You are only replacing the service provider side
14:08 poikilotherm
And that's totally up to you!
14:09 pdurbin
Yes but doesn't Harvard need to run an IDM like Unity or whatever?
14:09 poikilotherm
Of course, if those guys already run an IDM with OIDC, that would take a lot of work away from you
14:09 poikilotherm
Well, YOU need to run that, as your service provider
14:10 poikilotherm
If they do it for you, it obviously easier
14:10 pdurbin
Yeah, but you don't. You have identity people to run your OIDC IDM for you. No fair.
14:10 poikilotherm
You are running Shibboleth now as your Service Provider
14:10 pdurbin
donsizemore: does your IdP have OIDC?
14:10 poikilotherm
YOu need to replace that
14:11 pdurbin
It sounds like I should ask the Harvard IdP people for OIDC support. Is that right?
14:20 poiki joined #dataverse
14:21 poiki
Meh the Matrix bridge is down...
14:21 poiki
pdurbin: that depends. If you simply interface Dataverse with an OpenID Connect capable provider, you will most likely not have multiway auth
14:21 poiki
And that's most likely not what you want, right?
14:23 poiki
All this stuff is done with a middleware, in most cases called an IDM
14:23 poiki
Often also called "Identity broker"
14:24 poiki
You are free to choose which software you use for that, as long as you can hook into it with OIDC.
14:24 poiki
Prominent examples are Keycloak and Unity IDM, but most likely there are more.
14:25 poiki
IIRC the MIT folks have a Gluu server running
14:29 pdurbin
poiki: any plans to add an IDM like Unity or Keycloak to https://github.com/IQSS/dataverse-kubernetes ?
14:30 poiki
I don't think so
14:30 poiki
That's really beyond scope
14:30 poiki
There are already project out there you can use to setup those
14:30 poiki
+s
14:34 donsizemore
@pdurbin the library keeps talking about it; i can ask thu-mai and mandy
14:39 donsizemore
@pdurbin Q?
14:41 poiki joined #dataverse
14:41 poiki left #dataverse
15:40 jri_ joined #dataverse
15:44 jri joined #dataverse
15:46 pdurbin
donsizemore: thu-mai and mandy are OIDC hackers?
16:20 bricas
is it possible to run two dataverse instances on the same machine?
16:27 poikilotherm
bricas: with Kubernetes? Sure! :-D
16:28 poikilotherm
bricas: without kidding. You should be able to do that.
16:28 bricas
touche.
16:28 poikilotherm
I would look in the direction of multiple domains
16:28 poikilotherm
Because you will need to have different appserver envs
16:28 poikilotherm
Those will need to be configured for different ports etc
16:29 poikilotherm
So you will need a reverse proxy to deal with all the rewriting stuff etc
16:29 poikilotherm
Also the handling of asadmin gets a bit more complex
16:29 poikilotherm
And the installer etc is not ready for this in any way
16:30 poikilotherm
You'll be pretty much on your own
16:30 bricas
hrmmm. that's not great. :)
16:30 bricas
perhaps a new vm is the path of least resistance for now.
16:30 poikilotherm
But interesting experiment
16:30 poikilotherm
You might be interested in trying this on Payara
16:30 poikilotherm
Most likely that will be easier when it comes to getting support for any appserver issues you're running into
16:31 poikilotherm
GF 4.1 is dead
16:31 poikilotherm
Definitely
16:31 poikilotherm
That's why I love containers ;-)
16:35 bricas
we do have a k8s cluster now, actually. this just isn't part of it.
16:37 poikilotherm
:-D
16:37 poikilotherm
You could do your second installation on that :-D
16:38 poikilotherm
But be warned: no Shibboleth here.
16:38 bricas
oh. i need that.
16:38 poikilotherm
pdurbin (Mr. Shib) was flashed today
16:38 poikilotherm
https://github.com/IQSS/dataverse/issues/6701
16:38 poikilotherm
We run on K8s now and use OIDC for Shib login ;-)
16:39 poikilotherm
Or to be precise: to be able to use the eduGain SAML Federation. No Shib involved here.
16:39 poikilotherm
I am still tweaking the support, but we'll get there.
16:40 poikilotherm
That's what I say about Shib on K8s: https://dataverse-k8s.readthedocs.io/en/v4.19/day3/auth.html#saml-using-shibboleth-service-provider
16:43 poikilotherm
And as skasberger learned recently, joining Shib SP to a large federation is no fun.
16:45 bricas
okay. looks like a new vm wins out for now :)
16:46 poikilotherm
For my curiosity: what would be a key element for you to think about doing this on K8s?
16:51 bricas
well, if shib was able to work "normally" i'd give it a shot.
16:52 poikilotherm
He. OK. Would it help if you had a guide to setup an IDM like Keycloak as a replacement?
16:52 poikilotherm
(To replace Shib)
16:56 bricas
hard to say, not having known about keycloak until this very second. how does datavere talk to keycloak?
16:56 poikilotherm
OpenID Connect
16:57 poikilotherm
"The future" :-D
16:58 bricas
ah. we're actually setting up an "idp proxy" using simplesamlphp so we can avoid having to be at the mercy campus IT every time we add an app
16:58 bricas
wonder if there's any possibilities there
16:59 pdurbin
bricas: I've never tried to run two instances of Dataverse on the same machine if you don't count Vagrant. :)
16:59 pdurbin
poikilotherm: the version of Payara we intend to support was released 5 hours ago: https://github.com/payara/Payara/releases/tag/payara-server-5.201 /cc donsizemore
17:04 poikilotherm
https://github.com/simplesamlphp/simplesamlphp-module-openidprovider
17:04 poikilotherm
You should be able to use it as an identity broker
17:05 poikilotherm
Citing https://github.com/simplesamlphp/simplesamlphp-module-openidprovider/blob/master/docs/provider.txt: "This allows you to integrate OpenID into an existing IdP, or to add a bridge between OpenID and SAML 2.0."
17:09 poikilotherm
Alright guys, I'm outta here for today
17:09 poikilotherm
Read you all later....
17:17 bricas
poikilotherm: thanks for the info!
17:38 poikilotherm
pdurbin woohoo about Payara 5.201. it contains the JSF upgrade, so no manual patching anymore
18:30 pdurbin
Maybe I should switch my dev environment to Payara. Would I still need the installer? Can I just use the MicroProfile Config API ?
19:02 poikilotherm
pdurbin you will still need some script action to configure the domain... We need to change that for Dataverse 5. Have to send scolapasta an email about my idea of a breaking changes branch...
19:02 poikilotherm
Asked him at Tromso, but most likely he forgot ;-)
19:03 pdurbin
Hmm. Can you please create an issue about it?
19:22 poikilotherm
Sure. :-)
19:23 poikilotherm
Are there statistics for issues on GitHub?
19:26 pdurbin
Yes, please see http://science.osshealth.io/repo/New%20Repositories/dataverse/overview :)
20:47 donsizemore
@pdurbin pssst
21:20 pdurbin
donsizemore: sorry, was on an RO-Crate (Research Object Crate) call just now: https://s.apache.org/ro-crate-minutes (with Jim and Ana)... How can I help? :)
22:00 skasberger
@poikilotherm: it seems, as signature is making troubles. when we comment the MetadataFilterSignature line, it start immediately without problems. when its uncommented, the process takes forever (we mostly stopped it between 5 and 30 minutes after we started). No progress, no log information, no errors or warning wherever.
22:43 donsizemore
@pdurbin I think payara5, postgres10 and the python3-interpreted installer were too much for the rebuilt payara-5.201 today. I had a DB question but I want to fire off the job again tomorrow using the PERL installer before I ask in earnest
23:05 pdurbin
mmm, perl
23:05 pdurbin
my first love
23:34 aialves joined #dataverse
23:34 aialves
Hello
