Time
S
Nick
Message
02:09 jri joined #dataverse
05:09 jri joined #dataverse
08:17 jri joined #dataverse
10:39 juancorr joined #dataverse
13:34 dataverse-user joined #dataverse
13:46 pdurbin
hey there dataverse-user
13:46 dataverse-user
Hi
13:47 pdurbin
Is there anything we can help you with?
13:47 dataverse-user
How are you pdurbin? I'm installing a dataverse here at FGV (Brazil).
13:48 dataverse-user
They asked me to set-up the authentication to use our LDAP domain.
13:48 dataverse-user
So I've read that this may be possible by shibbolet http://guides.dataverse.org/en/latest/installation/shibboleth.html
13:49 dataverse-user
But I'm kind of lost on how to do it.
13:50 pdurbin
Well, LDAP via ORCID OAuth has been shown to work. Please see https://groups.google.com/d/msg/dataverse-community/9rb1VQeAKTU/7Lt-E8WkBgAJ
13:50 dataverse-user
We have two LDAP domains , and I need to figure out how to authenticate by both.
13:52 dataverse-user
Ok, but if some researcher does not have ORCID? He/she must have it? Right?
13:53 donsizemore joined #dataverse
13:54 dataverse-user
We are afraid of some resistance from the researchers.
13:54 dataverse-user
Resistance in use dataverse.
13:57 donsizemore
knock knock @pdurbin
13:59 drew-jhu joined #dataverse
14:34 pdurbin
donsizemore: hi, sorry, I was biking to work. It sounds like you want LDAP as a first-class option, similar to Shibboleth or OAuth. Is that right?
14:34 pdurbin
donsizemore: who's there?
14:36 pdurbin
hey drew-jhu thanks for the password alias pull request
14:37 drew-jhu
HTH. was actually hanging about to discuss @pameyer's feedback on it. since they're not around, maybe you can field a couple questions?
14:38 pdurbin
Sure, hit me.
14:39 drew-jhu
as you may have seen, they found an error with the unattended installer that my ansibilized use of the script doesn't hit...
14:40 pdurbin
yeah, I saw a comment about that
14:41 drew-jhu
i think i have a solution, that i am prepared to test in my environment, but it would be really helpful if someone who's better equipped could retest the unattended install script method, as well as any other install methods / environments that make use of the script
14:41 drew-jhu
is that a fair request? i'm down with testing my own contributions, but this particular piece is used in a lot of divergent situations
14:42 pdurbin
Hmm. Can I ask you to try it in Vagrant? It uses the unattended installer: https://github.com/IQSS/dataverse/blob/v4.10.1/scripts/vagrant/install-dataverse.sh#L29
14:43 drew-jhu
and also. we are cool with adding commits to PRs... no need to squash so 1 commit per PR?
14:44 pdurbin
We are absolutely cool with adding commits to PRs. I've had the problem where contributors will tell me "ok, try it now" when they've changed something in a pull request since I last tested it and there's no history of what they changed.
14:45 pdurbin
So I'd personally rather see the commits, the changes.
14:47 pdurbin
I'm happy to get you up to speed with Vagrant if you've never used it.
14:50 drew-jhu
i use vagrant all the time, but these days always with Ansible at the wheel. do you have instructions for the setup you'd like me to use?
14:51 pdurbin
I have a little write up at http://guides.dataverse.org/en/4.10.1/developers/tools.html#vagrant
14:51 pdurbin
if there's anything weird or wrong that I've written there, please feel free to fix it up in your pull request
14:52 pdurbin
oh and if you can fix https://github.com/IQSS/dataverse/issues/4519 while you're in there it would be great
14:52 drew-jhu
i see. you've a vagrantfile tucked in a pocket somewhere
14:53 pdurbin
well, it's in the root of the repo
14:53 drew-jhu
i can't tell if you're kidding about that last request
14:53 pdurbin
the vagrant destroy thing? I'd love a fix. no kidding
14:54 drew-jhu
you are good at your job
14:54 pdurbin
heh
14:54 pdurbin
so many bugs
14:55 pdurbin
I just finished reading Children of Time. Crawly things everywhere.
14:55 drew-jhu
i'll see what i can do. i do need to get back to other work, though, so...
14:56 pdurbin
Sure. My other thought is that Oliver is off for two weeks and would probably be happy to help with the password alias thing. I forget when he comes back.
14:58 drew-jhu
i used the script he shared to get started. it was helpful. i am 99% sure that the approach is fine, i'm just hitting a permissions issue in the specific location where i placed my temporary files. i'll be testing to see if putting them in /tmp will resolve the issue. (prolly where i should have put them in the first place, TBH)
14:58 pdurbin
Ok, sounds like you're not blocked and you're happy to keep hacking. Thanks!
14:59 drew-jhu
y. i just had questions about community norms, really
15:00 drew-jhu
and testing
15:02 pdurbin
It'll be tested by QA eventually but it sounds like it shouldn't go to QA quite yet.
15:02 pdurbin
Hmm, the QA column at https://waffle.io/IQSS/dataverse is empty though. But I wouldn't want to waste QA's time.
15:03 pdurbin
I can see other pull requests in code review that feel more ready for QA, to be honest.
15:03 pdurbin
That reminds me, thanks for weighing in on https://github.com/IQSS/dataverse/issues/5448 as well.
15:05 drew-jhu
not sure what the process is exactly. it passed CI. not sure how pameyer found the issue, but i'm grateful they did. guess there's something before QA?
15:05 pdurbin
pameyer is good at catching bugs
15:05 pdurbin
our CI... well... sometimes :)
15:06 pdurbin
I thought you said you wanted to switch to /tmp or something.
15:06 pdurbin
The "something before QA" is the developer saying "I'm done making commits". :)
15:09 drew-jhu
re: switch to /tmp... that's the change i just pushed and am about to test
15:10 pdurbin
ah, pushed already. ok
15:12 drew-jhu
so i'll respond to pameyer in the PR when i think it passes retesting, then i'll check back in
15:13 drew-jhu
thx for the help
15:13 pdurbin
Sounds fine. I was just thinking. I hope it doesn't break dev environments. Interns this summer or whatever.
15:14 pdurbin
Most developers run the installer once when they get a new laptop and never again.
15:16 drew-jhu
y. this fix does nothing for existing environments. i have an Ansible playbook that does, but it's heavily dependent on other parts of my environment (like the Ansible role I wrote to upgrade our Dataverse)
15:17 pdurbin
No I mean future environments. Future interns. Future contributors.
15:17 pameyer joined #dataverse
15:17 pdurbin
ah, there's pameyer now
15:17 drew-jhu
thanks for the catch on my PR, @pameyer
15:17 pameyer
still catching up on the logs, but I saw IT failures too
15:18 pameyer
no problem :)
15:18 pdurbin
pameyer: need moar CI for the installer :)
15:18 pameyer
am assuming the ITs failures were me misconfiguring stuff
15:18 pdurbin
pameyer: for when you go on vacation :)
15:18 pameyer
@pdurbin yup - assuming folks will remember to run scripts / push buttons is bad
15:19 pameyer
@drew-jhu was happy to see that PR, the less chance of folks having passwords posted in github issues the better :)
15:19 pdurbin
I wonder if the ec2 create script could have a mode where it uses the installer.
15:19 drew-jhu
@pameyer indeed. HTH
15:19 pdurbin
yes, I love the aim, fewer passwords in the logs
15:20 drew-jhu
this gets cleartext passwords out of domain.xml, as well
15:20 pdurbin
ah, good point
15:21 pdurbin
maybe it ties us to glassfish a little more but we're already quite bound to it
15:21 drew-jhu
can always revamp later, when the time comes
15:21 pameyer
if somebody malicious can read domain.xml, there's probably already worse problems
15:23 drew-jhu
well, there's always the situation where some helpful troubleshooter wants to see your configuration...
15:23 drew-jhu
or "helpful"
15:23 pameyer
:facepalm: right
15:24 drew-jhu
anyway, when getting two of them out of the log, it was easy enough to pick up the third
15:28 drew-jhu left #dataverse
15:29 drew-jhu joined #dataverse
16:10 dataverse-user
Thank you folks! I'll try to do as http://dataverse.ufabc.edu.br did.
16:11 pdurbin
dataverse-user: interesting. Are they using LDAP?
16:12 pdurbin
via Shibboleth?
16:13 pdurbin
or is it just plain Shibboleth?
16:18 dataverse-user
I'm trying to get in touch to discover
16:18 dataverse-user
I'm really new on this, sorry.
16:20 dataverse-user
By the way, my name is Julio.
16:20 pdurbin
dataverse-user: no worries. I was going to encourage you to create a "LDAP support" issue at https://github.com/IQSS/dataverse/issues because people keep asking about it.
16:21 pdurbin
"Julio" is probably taken but if you type something like /nick Julio1234 that nickname is probably not taken. Nice to meet you. :)
16:22 pameyer
I'm surprised there isn't an ldap support issue already
16:23 pdurbin
Me too, given how often people ask.
16:24 pdurbin
It should be too crazy hard to extend CredentialsAuthenticationProvider for at least a single LDAP server. Not sure about multiple. dataverse-user says he has two LDAP domains.
16:27 pameyer
it looked like ldap -> shib might be workable. but I haven't looked at it much, and probably should wait until I've gotten a working shib setup to assume any of it makes sense to me
16:30 pdurbin
pameyer: wait, do you want LDAP support too?
16:33 pameyer
pdurbin: partly just evaluating alternatives, and partly trying to map shib onto something I was closer to understanding
16:34 pdurbin
I think if Shib as a more centrally controlled OAuth where users aren't empowered as much. If that helps. :)
16:34 pdurbin
"We will decide whether or not we will give out your email address."
16:35 pdurbin
Julio010101: it worked! Yay IRC !
16:35 * pdurbin
dances
16:35 pameyer
thinking of shib as a way to hook up apache to a ldap-like source and pass it on to glassfish seemed like a reasonable analogy
16:37 pdurbin
There was the idea from Kristi of making it all consistent by using environment variables.
16:38 pdurbin
He suggested https://github.com/zmartzone/mod_auth_openidc at https://github.com/IQSS/dataverse/issues/4383#issuecomment-363191809
16:41 Julio010101
There are other applications here that use OAuth2 to communicate with LDAP. I scheduled a meeting here next Monday to understand how does it work.
16:44 pameyer
Julio010101: do you know which LDAP servers you'll need to use? freeipa?
16:44 pdurbin
Julio010101: cool, but which OAuth2 providers? Google? Unfortunately, Dataverse is hard coded right now to only work with three OAuth providers: Google, GitHub, and ORCID. If you look at #4383 you'll see that we'd like to make it easier to add more, to add arbitrary OAuth providers.
16:45 Julio010101
Our LDAP is from Microsoft
16:46 Julio010101
It's a windows domain.
16:46 Julio010101
@pdurbin, understood!
16:46 pameyer
ADFS?
16:46 Julio010101
Yes!
16:47 pameyer
I've been working on that as well
16:48 donsizemore
@Julio010101 a guy from WCU got Shibboleth talking to UNC's Active Directory with some attribute mapping.
16:48 donsizemore
@Julio010101 I have our half of the file, he never would send me the pertinent bits from the AD side =)
16:49 pameyer
@donsizemore good to hear
16:49 Julio010101
@donsizemore, it's exactly what I need.
16:49 pameyer
@Julio010101 I don't have a working setup yet, but no signs it'll be impossible
16:50 donsizemore
i can probably send you both a redacted copy of the Shibboleth side without angering the guy who did it.
16:50 Julio010101
It would be nice.
16:51 pameyer
thanks - that should be helpful. my guess is that we may have attribte mapping for eppn set incorrectly; but I'm deferring to the local AD expert at the moment
16:52 donsizemore
i'm removing the certs and identifiers... i'll send you the sliced up copy (via email?)
16:52 Julio010101
@pameyer, understood. As soon as I have more details, I'll share here.
16:52 Julio010101
my be, julio.chaves
16:53 pameyer
@donsizemore however's easiest for you
16:53 Julio010101
may be, I meant
16:54 donsizemore
on the way. i think i pulled out all the state secrets
16:54 pameyer
I've been accumulating notes intended for a doc PR; but until it works that's a little premature
16:54 pdurbin
Do you ADFS folks think something should go in the Dataverse guides about this? If so, can someone please create an issue?
16:55 Julio010101
@pdurbin, since I get the things more clear to me, I'll open an issue, for sure.
16:56 pdurbin
thanks!
16:56 Julio010101
I must accomplish this setup in order to implement dataverse here
16:57 pdurbin
Makes sense. Auth is important.
18:07 andrewSC joined #dataverse
19:02 pameyer
Julio010101 , donsizemore : still non-functional with shib/adfs , but it seems like things are going in the right direction
19:08 drew-jhu
@pdurbin retested revised PR with & w/o ansible, as requested. added comment to the PR to that effect & i can see it with status "Code Review" in your waffle. i also looked into switching the vagrantbox to the official Centos 7 box, in an attempt to address the "vagrant destroy" issue, as also requested. no dice. (that change also churned up a pile of other issues, unfortunately). as others have already reported, the fix seems to be comment
19:10 pdurbin
drew-jhu: hi! Sorry, you were cut off. "fix seems to be commenti"?
19:10 drew-jhu
as others have already reported, the fix seems to be commenting in the standalone.vm.box value. i also encountered no problems when leaving it in place.
19:12 pdurbin
commenting in. right. but if memory serves, we can't leave it in
19:14 drew-jhu
that was asserted, w/o elaboration. a later comment in that issue reported no such issues. i also didn't encounter issues when i left it in (for the hour or so i've been using it)
19:14 drew-jhu
what i can tell you is that it doesn't look like switching your base box will fix it
19:16 pameyer
@drew-jhu 103eeff4a7b4076e8c8e3f68b59b8efa24f91247 fixed the problems I saw with the non-interactive install warnings
19:16 pdurbin
Ok, so maybe we can just uncomment that line in a pull request. Feel like making that PR? :)
19:17 pdurbin
pameyer: no objections if you want to move it to QA.
19:17 drew-jhu
@pameyer thanks for testing for me!
19:18 pameyer
5487? I don't want to move something to QA if I can't get it to pass ITs
19:18 pameyer
not to discourage anyone else from doing that
19:18 pdurbin
pameyer: oh! I thought you said it was passing now. My mistake!
19:19 pameyer
non-interactive install warnings/errors I mentioned are fixed :)
19:19 pdurbin
ok, but the API test suite fails for you?
19:19 pameyer
@drew-jhu sorry I've been multi-tasking on that; but happy to help
19:20 drew-jhu
@pdurbin you're a great motivator, but this time, i'm going to have to decline. i really am overdue on other projects with fast-approaching deadlines. also, i feel less comfortable making PRs on things I don't really use (like your vagrantfile)
19:20 pdurbin
drew-jhu: no worries, thanks for poking at it :)
19:21 drew-jhu
HTH
19:52 pdurbin
drew-jhu: from another quick look at your pull request, I don't think it will break new dev environments. I used "trust" for postgres and "FAKE" as my PID provider.
19:53 drew-jhu
i don't think it will break anything either
19:54 pdurbin
any reason you switched from double to single quotes?
19:56 drew-jhu
yes. needed to convey the literal string ${ALIAS=doi_password_alias} (etc) through the script to the properties in question
19:58 pdurbin
ah, ok
19:58 pdurbin
makes total sense then
19:58 pdurbin
I really appreciate you digging into this.
19:59 drew-jhu
thx for coming up with the technique
19:59 pdurbin
well, it was some dude in #glassfish
19:59 pdurbin
Techni or something
20:00 pdurbin
all I did was rattle some cages :)
20:00 pdurbin
nice guy
20:00 pdurbin
he's helped out before
20:00 drew-jhu
an essential step
20:01 pdurbin
yeah
20:02 pameyer
pdurbin: API test suite was failing for me on that branch; but that's almost certainly because I didn't update the configuration scripts
20:02 pdurbin
Ooooooh. Hmm.
20:02 pdurbin
Which scripts?
20:03 pameyer
configure_doi
20:03 pdurbin
gotcha
20:03 pameyer
with the ezid -> datacite switchover in the installer, I needed to add something
20:03 pameyer
at least, there was a time window when I did - not sure if it's still needed
20:05 pdurbin
ok, so you set some environment variables. stuff like ${doi_password}
20:06 pdurbin
ARG doi_password=apitest in conf/docker-aio/c7.dockerfile. interesting
20:06 pameyer
yeah - but somewhere in there is a asadmin set-jvm-option that needs to get switched out with an alias equivalent
20:06 pdurbin
or you could switch to "FAKE" :)
20:06 pameyer
and when I did the asadmin commands for the alias manually, that's when I saw the API tests failing
20:07 pameyer
I thought I did switch the provider to FAKE at some point
20:07 pdurbin
oh, looks like you did
20:07 pdurbin
but still no worky? weird
20:08 pameyer
definately weird
20:08 pameyer
I'm pretty sure it was either a glitch, or me typoing something - not that branch
20:09 pdurbin
but I'm likely to hit https://github.com/IQSS/dataverse/issues/5374 if I try, right?
20:10 pameyer
yeah :( `docker -exec it bash dv ` ; `cd /usr/local/glassfish4 ; bin/asadmin start-domain --debug`
20:10 pdurbin
:(
20:10 pdurbin
I should probably give it a whirl. It's been a while.
20:11 pameyer
... typos again - `docker exec -it bash dv`
20:12 pdurbin
so I should try it on "develop" first
20:13 pameyer
develop was working ok for me (other than that glitch) earlier this week
20:14 pdurbin
oh! wait, so it only fails some of the time?
20:14 pameyer
no, it fails consistently. but other than needing to start glassfish manually, everything else seems to work the way its supposed to
20:15 pdurbin
ok, so the "plumbing" problem is glassfish not being up
20:15 pameyer
yup
20:16 pdurbin
Ok. Well, I'm running it now. I've had way worse plumbing problems at home. You never want to hear the words "soil stack".
20:21 pdurbin
"seturl fail; bailing out"
20:21 pameyer
seturl doesn't work with no glassfish :(
20:22 pameyer
broken plumbing means post-deploy config scripting needs to be run manually. seturl and configure_doi
20:27 pdurbin
Huh, and glassfish stops after running this, which is why you add --debug, maybe: docker exec -it dv /usr/local/glassfish4/bin/asadmin start-domain
20:29 pameyer
I think glassfish stopping isn't related to --debug ; that's just so I can hook it up to jdb if necessary
20:43 pameyer
pdurbin: do you remember offhand if the "sn" in shibboleth is surname?
20:47 pdurbin
yep, surname
20:48 pameyer
cool - thanks
20:55 pameyer
initial indications are that ADFS, shibboleth and dataverse are capable of playing nicely with one another
20:56 pdurbin
Julio010101: ^^
20:59 pdurbin
pameyer: great news
21:03 pameyer
yup - next problem is to figure out how to document it
21:07 pameyer
also makes me wonder if that means ldap would work
21:07 pdurbin
does it use LDAP?
21:09 pameyer
:shrug: possibly mistaken impression from earlier conversation (without checking the logs)
21:09 pdurbin
hey, as long as it works :)
21:18 pdurbin
drew-jhu: I move your password aliases pull request to QA but I left a comment in the issue about a doc (guides) update. I hope it makes sense.
21:18 pdurbin
moved*
21:18 pdurbin
https://github.com/IQSS/dataverse/issues/5412#issuecomment-457727815
21:44 pdurbin
anyway, nice chat today all. have a good weekend!
21:44 pdurbin left #dataverse
21:44 drew-jhu
thanks, @pdurbin
21:48 drew-jhu left #dataverse
22:02 sivoais joined #dataverse
