Time
S
Nick
Message
08:18 juancorr joined #dataverse
08:19 jri joined #dataverse
09:44 stefankasberger joined #dataverse
09:52 poikilotherm joined #dataverse
10:22 juancorr joined #dataverse
10:43 MrK joined #dataverse
12:08 pdurbin joined #dataverse
12:10 pdurbin
I hope everyone had a nice weekend. :)
12:49 donsizemore joined #dataverse
13:53 ashaw joined #dataverse
14:00 ashaw
hi all - we are running Dataverse and at the end of last week, the dataverse api was successfully creating new datasets, but the attachments were not coming through. Glassfish logs showed " alreadyExists failed" and "String edu.ucsb.nceas.ezid.EZIDException: bad request - invalid identifier". Would I be correct in assuming this a result of never switching from EZID to Datacite?
14:16 pdurbin
I'm not sure why attachments would matter. Weird.
15:25 ashaw joined #dataverse
15:26 ashaw
I know - it was only the file attachments that were not coming through. I still should be updating Glassfish to use Datacite, is that correct?
15:30 pdurbin
ashaw: yeah, unless you happen to be at the University of California (which still allows EZID to be used), you need to switch to DataCite. You can get a discount on DataCite DOIs by joining the Dataverse consortium, if you'd like.
15:34 ashaw
pdurbin: thanks, that is probably what we'll do.
15:35 pdurbin
ashaw: great! Here's the website: http://dataversecommunity.global
15:35 pdurbin
ashaw: is your installation already on the map at https://iqss.github.io/dataverse-installations/ ?
15:47 ashaw
pdurdin: we haven't published our dataverse yet -- all our entries are draft status until we publish publicly.
15:47 ashaw
pdurbin: is it possible to configure dataverse so it generates the doi but does not go through the doi registration process?
15:50 pdurbin
ashaw: do you mean real DOIs or fake DOIs?
16:05 ashaw
pdurbin: (I am new to this so I hope get this correct). Our dataverse installation has been generating DOIs, but it looks like those were not officially registered with EZID. So, believe I am looking to continue generating fake DOIs.
16:10 pdurbin
ashaw: no worries. We were all new at this once. :) You could play around with the "FAKE" DOI provider: http://guides.dataverse.org/en/4.18.1/installation/config.html#doiprovider . What version of Dataverse are you running?
16:29 juancorr15 joined #dataverse
16:42 ashaw
pdurbin: is there an easy way to check the version?
16:42 ashaw30 joined #dataverse
16:44 ashaw joined #dataverse
17:49 pdurbin
ashaw: you could try something like this: https://demo.dataverse.org/api/info/version
17:49 pdurbin
It should also be in the bottom right of the home page.
18:16 nightowl313 joined #dataverse
18:18 ashaw
pdurbin: looks like: {"status":"OK","data":{"version":"4.8.5","build":"139-64cb1ef"}}
18:20 ashaw
pdurbin: it looks like we have always been in testing mode as the shoulder "FK2/" has been used in our DOI
18:21 nightowl313
pdurbin: it's me ... the one trying to get shibboleth to work! i'm a bit confused...I am using the sample attribute-map.xml that you have here: https://dev2.dataverse.org/guides/build/html/installation/shibboleth.html#attribute-map-xml and it doesn't look like "mail" is commented out. I looked at the github issue last week where you talked about uncommented those lines, but they were not commented in my file.
18:22 pdurbin
ashaw: gotcha. You'd need to upgrade to Dataverse 4.10 or higher to use the "FAKE" DOI provider. Also, we recommend real DOIs (or Handles) if that's a possibility. Please see https://github.com/IQSS/dataverse/issues/6423 :)
18:23 pdurbin
nightowl313: cool. Thanks for switching to my sample file. Did you also restart shibd and httpd?
18:23 pdurbin
donsizemore: Mr. Shib is getting back into it. :)
18:24 nightowl313
i was using it last week...I will try restarting the services again
18:24 pdurbin
great
18:25 pdurbin
nightowl313: ok, I couldn't remember if I linked to that brand new version of the Shib page last week or not. I made my pull request late Wednesday.
18:26 nightowl313
yea, still just goes through the process of redirecting to my SSO, DUO, etc. and then returning to the DV "Account Information" page, as if it has logged me in, but I am not logged in, and the account is not created
18:26 nightowl313
hmmm...I will try to replace the file again...I compared the one I am using with the one on the page now, and they look the same
18:26 pdurbin
Anything in server.log?
18:27 nightowl313
still says "The SAML assertion for "Shib-Identity-Provider" was null. Please contact support."
18:27 pdurbin
Oh! Huh.
18:29 pdurbin
nightowl313: do you want to try it with https://samltest.id ? That's what my pull request was about.
18:30 nightowl313
do I use the "Test Your SP" option?
18:31 pdurbin
nightowl313: yes but before you dive into that... I *just* rebuilt the docs at https://dev2.dataverse.org/guides/build/html/installation/shibboleth.html . It didn't have the samltest.id stuff there yet. Sorry. Now it does.
18:31 pdurbin
Which means you'd need to re-download the attribute map xml file again if you want to use it.
18:32 pdurbin
and the "sample dataverse-idp-metadata.xml file" I offer for download is for samltest.id
18:32 pdurbin
so you'd have to redownload that too
18:33 pdurbin
Yeah. Those are the two XML files I updated.
18:34 pdurbin
And yeah, what you're setting up is an SP (Service Provider) not an IdP (Identity Provider).
18:35 pdurbin
I just think it might be nice for you to log in with Rick or Morty to see it all working. :)
18:38 nightowl313
okay, will give it a try...thanks for your help! One thing that I have noticed is that, the doc says that, after installing shibboleth, both /etc/httpd/conf.d/shib.conf and /etc/httpd/conf.d/shibboleth-ds.conf should be present. I only see shib.conf and not shibboleth-ds.conf...does that matter?
18:40 pdurbin
Hmm, let me check my server.
18:42 pdurbin
nightowl313: no shibboleth-ds.conf on my server. Do you mind creating an issue at https://github.com/IQSS/dataverse/issues to start tracking stuff like this? You could just call the issue "shib doc issues" or something.
18:43 nightowl313
sure...but you don't have one either? it is related to the shibboleth discovery service...so thought it might be needed
18:43 nightowl313
but you don't have it either...so I'll just put an issue related to the docs saying it should be there
18:44 pdurbin
but my discover service is working fine so I don't think it's needed
18:44 pdurbin
donsizemore might know :)
18:50 pdurbin
discovery*
19:06 nightowl313
so, to do the SAMLtest, I replaced attribute-map.xml with the sample one that you provided on the Shibboleth doc, and I replaced the dataverse-idp-metadata.xml file with the samltest one, then I re-added shibboleth as a provider to dataverse. I went to my DV login page and saw samltest as a provider...clicked it and logged in as "Rick"...it still takes me to the "account information" page but does not log me in or create rick's account.
19:07 pdurbin
nightowl313: d'oh! I'm glad we're doing this though. We can leave your poor identity people alone for a moment. :) Did you check "/Session" ?
19:11 nightowl313
yes, it has all the things...lists displayname, eppn, givenname, mail, sn, telephone, uid
19:11 pdurbin
huh
19:12 pdurbin
and the same error in server.log about shib id provider?
19:12 nightowl313
no error in the log
19:13 nightowl313
on no, sorry, same error in the log about the identity provider
19:14 pdurbin
well, at least it's consistent :)
19:14 pdurbin
If we fix it for samltest.id it should be fixed for your pilot. :)
19:14 nightowl313
and, this error: "No mime type could be found for file bs/css /bootstrap.min.css.map" just a warning
19:15 pdurbin
ok, I'm not too worried about that bootstrap warning
19:15 pdurbin
If you run rpm -q shibboleth do you get something like shibboleth-3.0.4-3.2.x86_64?
19:16 ashaw
pdurbin: thanks! short of upgrading right now (which is planned for next year) is it possible to for me to continue generating, essentially test/fake DOIs as I have been doing, except switching the glassfish baselineURL from ezid to datacite
19:17 ashaw
pdurbin: trying to find an easy workaround for why uploads are suddenly failing
19:17 pdurbin
ashaw: well, you could contact DataCite and ask for account for testing.
19:18 nightowl313
yes shibboleth-3.0.4-3.2.x86_64
19:19 pdurbin
Ok. And you're using shibboleth2.xml from the Dataverse guides, right?
19:20 nightowl313
yes
19:20 pdurbin
And a relatively recent version of Dataverse?
19:22 pdurbin
All I see in server.log is "no user found using rsanchez
19:22 pdurbin
nightowl313: I'm talking about https://dev2.dataverse.org if you'd like to try it.
19:24 nightowl313
yes, I see yours...mine just shows a blank "Account information" page ... my site should be live so you can try it... can you get to https://dataverse-dev.lib.asu.edu ?
19:25 pdurbin
Tried it. Blank page, like you said. I wonder what's different. Hmm.
19:26 pdurbin
https://dataverse-dev.lib.asu.edu/Shibboleth.sso/Session looks almost identical to what I posted at https://github.com/IQSS/dataverse/issues/6018#issuecomment-559306722
19:27 pdurbin
including this... Identity Provider: https://samltest.id/saml/idp
19:28 nightowl313
does it have anything to do with the fact that I am getting my SSL certs from AWS ACM? That seems to be the thing that is different from everyone else.
19:28 pdurbin
I doubt it.
19:29 nightowl313
I used dataverse-ansible to install everything
19:29 pdurbin
We could try turning up the debugging in ShibUtil, I guess.
19:29 nightowl313
went through like a charm...no issues at all
19:32 pdurbin
nightowl313: can you please try this: /usr/local/glassfish4/bin/asadmin set-log-levels edu.harvard.iq.dataverse.authorization.providers.shib.ShibUtil=FINE
19:33 pdurbin
and then log in again?
19:33 pdurbin
afterwards, in server.log, I see this extra debugging information: shib values: [Shib-Identity-Provider: https://samltest.id/saml/idp , eppn: rsanchez
19:35 nightowl313
the log shows shib values: []|#]
19:35 nightowl313
it doesn't see any of them
19:36 nightowl313
going to copy your shibboleth2.xml again to make sure I didn't mess anything up in the one that I am using
19:38 pdurbin
nightowl313: if you do "grep AJP /etc/shibboleth/shibboleth2.xml" do you see 'REMOTE_USER="eppn" attributePrefix="AJP_">'?
19:40 nightowl313
yes
19:40 nightowl313
and I tried re-copying the shibboleth2.xml and it didn't change anything
19:41 nightowl313
changed "entityID="https://dataverse-dev.lib.asu.edu/sp " but everything else the same
19:42 pdurbin
Hmm. it's been a while but the behavior you're describing remind me of when 'attributePrefix="AJP_"' is missing.
19:43 nightowl313
I see it there... argh...I have to run to a meeting...thank you sooooooooo much for your help...I will pick it back up as soon as the meeting is over...
19:43 nightowl313 left #dataverse
19:44 pdurbin
Yeah, when I take out that "AJP_" thing I get the same behavior. "shib values: []" followed by 'The SAML assertion for "Shib-Identity-Provider" was null. Please contact support.'
19:46 pdurbin
... blank "Account Information" screen, etc. I'm pretty sure that's it. AJP_ is very much required.
19:49 pdurbin
ashaw: sorry, I've been troubleshooting that login issue since last week. You're saying you can't even add files to an existing dataset?
19:51 ashaw
pdurbin: no worries! let me recap what the issue is just so you are clear
19:51 pdurbin
ok
20:08 bjonnh joined #dataverse
20:10 pdurbin
xarthisius: did you see some feedback came in on https://github.com/jupyterhub/binderhub/pull/969 ? Thanks again for that pull request!
20:13 ashaw joined #dataverse
20:13 ashaw
pdurbin: we have a web application that hits dataverse creates and entry and uploads the corresponding file/files. Middle of last week, we started getting three GlassFish Errors "edu.harvard.iq.dvn.core.index.DOIEZIdServiceBean/ alreadyExists" "String edu.ucsb.nceas.ezid.EZIDException: bad request - invalid identifier" and "edu.harvard.iq.dvn.core.index.DOIEZIdServiceBean localized message bad request - invalid identifier". Our application create
20:14 ashaw
the entry and then uploads the file. The strange part is that depending on the upload file type, you will get different results. For example, .pdf files do not upload. .pptx files upload but are corrupted. text files do not upload.
20:16 ashaw
that's correct, we create a dataset, and then immediately add the files...so the dataset is existing at the time of upload
20:19 pdurbin
ashaw: ok. And did anything change on your end? I thought EZID pulled the plug a long time ago.
20:22 ashaw
pdurbin: we made no changes to our web app, dataverse, etc.
20:25 pdurbin
Ok. Maybe EZID pulled the plug. Hard to know.
20:25 pdurbin
I guess another option you have is switching to Handles. I've never set up a Handle server though.
20:26 pdurbin
It seems like you probably need to get off EZID.
20:26 pdurbin
Real DOIs from DataCite would probably be the best path forward. But I understand you're looking for a short term work around as well.
20:27 pdurbin
If I were you I would probably ask DataCite for a test account. I'm not sure how long the turn around time is, though.
20:27 ashaw
pdurbin: yeah I reached out to EZID to try to confirm when the plug was pulled. have a request into Datacite as well. really appreciate your help on this
20:28 pdurbin
Sure. Sounds like a frustrating situation. And you definitely can't upgrade Dataverse sooner rather than later? We're happy to answer questions, etc.
20:32 ashaw joined #dataverse
20:34 ashaw
pdurbin: the upgrade route may be possible sooner. i see in the doc that updates need to be done sequentially, does this mean going from 4.8 to 4.10 would require 4.9 in addition to the minor revisions?
20:36 pdurbin
ashaw: well, there's a new somewhat experimental way to avoid those in between releases. Let me look it up.
20:38 pdurbin
ashaw: please see "A note on upgrading from older versions" at https://github.com/IQSS/dataverse/releases/tag/v4.12
20:39 pdurbin
here's the readme: https://github.com/IQSS/dataverse/blob/v4.18.1/scripts/database/README_upgrade_across_versions.txt
20:40 pdurbin
about dbupgrade.sh
20:40 pdurbin
(the experimental script, which did go through QA at least) :)
20:41 ashaw
pdurbin: awesome. let me look through this and follow up with any questions
20:41 pdurbin
And once you get to Dataverse 4.12 database migrations are handled by Flyway. So hopefully upgrades will be easier.
20:41 pdurbin
sounds good
21:19 nightowl313 joined #dataverse
21:31 nightowl313 joined #dataverse
21:50 nightowl313 joined #dataverse
21:54 nightowl313
pdurbin: i am running dv 4.18
21:54 nightowl313
i never answered you
21:55 nightowl313
wonder if there is something not working with AJP ... if I run ./asadmin list-network-listeners it is listed, but could something else be configured incorrectly with it?
22:06 poikilotherm joined #dataverse
22:21 pdurbin
Hmm, I guess you could check the proxy settings in ssl.conf
22:34 nightowl313 joined #dataverse
22:36 nightowl313
pdurbin: I have the section added to my original ssl.conf that was suggested in your shib documentation; i didn't replace that file, though
22:49 nightowl313
however, I just replaced mine with that one, and non change
22:49 nightowl313
no change
22:55 pdurbin
nightowl313: hmm. And shibboleth2.xml is correct and you've restarted shibd followed by httpd?
22:57 nightowl313
yep...so very weird...I'm at a loss
22:58 nightowl313
i'm using the sample idp-metadata, sample shibboleth2.xml, the metadata that you linked for testSAML, and the sample ssl.conf
23:36 nightowl313 left #dataverse
23:49 nightowl313 joined #dataverse
