Time
S
Nick
Message
08:34 cdsp-rmo joined #dataverse
08:34 cdsp-rmo
pdurbin: aaaaaand it's gone :'(
08:35 cdsp-rmo
I created a dataverse (I had to log multiple times to access the root page, then for creating the dataverse, strange)
08:35 cdsp-rmo
then I wanted to create a dataset in the dataverse
08:35 cdsp-rmo
when I clicked on "create a dataset", I had the publishing error
08:36 cdsp-rmo
so I told to myself "oh right, the siteUrl is not defined, let's define it"
08:36 cdsp-rmo
so I created it, then stopped glassfish, start glassfish
08:36 cdsp-rmo
and the previous error "can't log in french" appeared
08:37 cdsp-rmo
(I did the command "LANG=C mvn package" to build the war when I deployed yesterday)
09:50 tcoupin joined #dataverse
10:44 pdurbin
:(
10:51 tcoupin
Hi everyone
10:52 tcoupin
To deploy DV in my institut, user needs to be able to provide an existing DOI when create dataset
10:55 tcoupin
I started develop and have now an input to provide DOI on dataset creation
10:55 pdurbin
tcoupin: hi! If you're on "develop", you can try this new "Import a Dataset into a Dataverse" API endpoint: https://github.com/IQSS/dataverse/blob/ddc41f58c1cc21449eb5d1f9a69a13be7915ca8d/doc/sphinx-guides/source/api/native-api.rst#import-a-dataset-into-a-dataverse
10:57 tcoupin
ok
10:58 tcoupin
currently I can provide an existing DOI but publish the dataset fail
10:59 tcoupin
What is the purpose of "identifierRegistered" in dvobjects ? Can I use it to avoid publishing the doi ?
11:03 pdurbin
When publishing fails, is there a stacktrace in server.log?
11:04 pdurbin
I'm looking at "identifierregistered" at http://phoenix.dataverse.org/schemaspy/latest/tables/dvobject.html and from the name I assume it switches from false to true after a dataset has been published. Not sure.
11:15 tcoupin
I use the import api with an existing doi whitout release paramater
11:17 tcoupin
I cant publish the imported dataset, with same error "his dataset may not be published due to an error when contacting the <a href=https://ezid.cdlib.org target="_blank"/> EZID </a> Service. Please try again."
11:18 tcoupin
On auto release with api, a boolean is added to the command to prevent making doi public
11:21 tcoupin
when i create a dataset, identifierRegistered is true before publish
11:21 tcoupin
false for harvested dataset
11:22 tcoupin
i assume identifierRegistered is true when identifier was created by dataverse installation
11:24 pdurbin
You have to be in California to use EZID: https://www.cdlib.org/cdlinfo/2017/08/04/ezid-doi-service-is-evolving/
11:25 pdurbin
Instead of EZID, you will need to select another persistent id provider: https://github.com/IQSS/dataverse/blob/ddc41f58c1cc21449eb5d1f9a69a13be7915ca8d/doc/sphinx-guides/source/installation/config.rst#persistent-identifiers-and-publishing-datasets
11:30 tcoupin
Publishing work for dataset whitout existing doi
11:32 pdurbin
Ah, that's good.
11:32 pdurbin
EZID still works outside California today but soon it will stop working.
11:33 pdurbin
It sounds like you may have found a bug with that new "import" code.
11:33 pdurbin
Right? Does it seem like a bug?
11:37 pdurbin
If there is something to fix or documentation to clarify, please open a GitHub issue.
11:41 tcoupin
yes bug => https://github.com/IQSS/dataverse/issues/5104
11:52 cdsp-rmo joined #dataverse
11:52 tcoupin
seems strange : https://github.com/IQSS/dataverse/blob/ddc41f58c1cc21449eb5d1f9a69a13be7915ca8d/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/PublishDatasetCommand.java#L51
12:00 cdsp-rmo
hello tcoupin
12:00 cdsp-rmo
have you set -Ddataverse.siteUrl ?
12:00 cdsp-rmo
(jvm-options)
12:01 cdsp-rmo
pameyer told me about that here https://github.com/IQSS/dataverse/issues/4593#issuecomment-417677824
12:01 cdsp-rmo
it seems it is needed to publish
12:02 tcoupin
not set
12:04 cdsp-rmo
I have a question for you on my side
12:05 cdsp-rmo
about the login problem ...
12:05 cdsp-rmo
you don't have the problem anymore ?
12:05 tcoupin
no
12:06 cdsp-rmo
simply by doing "LANG=C mvn package" ? Then deploy it back on glassfish, stop and restart ?
12:06 cdsp-rmo
(have you stop and start glassfish later since the moment it worked ?)
12:09 tcoupin
glassfish was restarted more than one time ;)
12:10 tcoupin
glassfish is running in a docker container so locale is en too
12:10 tcoupin-test joined #dataverse
12:13 cdsp-rmo
that's strange then
12:14 cdsp-rmo
because it worked yesterday (after a laborious process), but when I restarted glassfish today, pouf, the problem was back ...
12:14 cdsp-rmo
I think dataverse hates me :D
12:15 tcoupin17 joined #dataverse
12:16 tcoupin joined #dataverse
12:17 donsizemore joined #dataverse
12:19 tcoupin
cdsp-rmo: available for a video using rendez-vous by renater ?
12:21 tcoupin joined #dataverse
12:24 cdsp-rmo
hum
12:25 cdsp-rmo
never used that
12:25 cdsp-rmo
that would use ful, but the thing is my working env is on a fix computer, and I'm in an open space
12:25 tcoupin
https://rendez-vous.renater.fr/4fp6n-asy6s-yrlfw
12:26 cdsp-rmo
so, not really indicated for mic or else
12:26 tcoupin
yes
12:26 cdsp-rmo
but if it's not a problem to use a chat, then it's ok
12:26 tcoupin
phone perhaps ?
12:27 cdsp-rmo
I don't have a phone
12:27 cdsp-rmo
:D
12:28 cdsp-rmo
is it really a problem to not be on vocal ?
12:31 cdsp-rmo
(sorry to be such a pain, but my office is not the best for conf calls :S )
12:43 tcoupin
no pb
12:51 cdsp-rmo
I think I will handle the login problem by over riding it in the code
12:55 tcoupin
cdsp-rmo: what have you done on my laptop ??? why is my dataverse in french now ??
12:57 Jim64 joined #dataverse
13:01 cdsp-rmo joined #dataverse
13:01 cdsp-rmo
wait
13:01 cdsp-rmo
what ?
13:02 tcoupin
i restart my firefox, and dataverse was in french after new deploy
13:03 cdsp-rmo
0o
13:03 cdsp-rmo
well, I do have my application in french since I used the latest develop branch
13:03 cdsp-rmo
not you ?
13:05 tcoupin
i force firefox to use en with "Quick Accept-Language Switcher"
13:05 tcoupin
but it seems to be session dependant
13:08 Jim54 joined #dataverse
13:11 Jim54
Hi all! Is there any guidance out on moving existing files to s3? It looks like I should just sync my files/authority dir(s) with the bucket and prepend s3://<bucketname>: to the DataFile storageidentifiers. Is that correct? Anything else (aside from having s3 set up in general)?
13:15 cdsp-rmo joined #dataverse
13:16 Jim__ joined #dataverse
13:18 cdsp-rmo joined #dataverse
13:21 tcoupin
cdsp-rmo: when I open dataverse in chrome (no plugin to change locale), DV is in french BUT label in login form are in english
13:22 cdsp-rmo
yes !
13:22 cdsp-rmo
I realized this
13:22 cdsp-rmo
very strange
13:22 cdsp-rmo
hum
13:22 cdsp-rmo
for the login page I have "nom d'utilisateur" and "mot de passe", but the rest is in english
13:22 cdsp-rmo
I "solved" my problem by hardcoding my password in passwor tests
13:23 cdsp-rmo
so I can do my dev without troubles with the rest, waiting for the thing to be solves ^^
13:23 cdsp-rmo
you can still login ?
13:23 tcoupin
on login page, title is "se connecter" but labels are username/password
13:23 cdsp-rmo
Or you can't anymore ? Is it just the french language bothering you ?
13:23 tcoupin
and login work
13:24 cdsp-rmo
damned
13:25 cdsp-rmo
do you have troubles to create dataset ? It seems my ezid is badly configured
13:26 cdsp-rmo
do you use the apitest account for ezid ?
13:27 tcoupin
https://github.com/IQSS/dataverse/issues/5104
13:28 cdsp-rmo52 joined #dataverse
13:28 cdsp-rmo52
I saw your troubles for publishing
13:28 cdsp-rmo52
but I can't even create a dataset on my side
13:28 cdsp-rmo52
manually, I mean
13:30 tcoupin
http://irclog.iq.harvard.edu/dataverse/2018-09-27#i_73814
13:32 cdsp-rmo52
yes, but right after that, he said it still works ><
13:35 cdsp-rmo52
also, how are we supposed to use dataverse as dev if the ezid doi is down ? I can't found a datacite api test account around
13:47 pdurbin
Jim__: hi! You had a question about moving to S3?
13:48 pdurbin
cdsp-rmo52: tragic that your dev environment is broken again after all that work yesterday :(
13:49 pdurbin
donsizemore: tell us more about the job ad you posted: http://irclog.iq.harvard.edu/dataverse/2018-09-26#i_73787
13:49 pdurbin
tcoupin: thanks for opening that issue about publishing
13:50 cdsp-rmo52
well, I decided to comment the login thing and now the doi call
13:50 cdsp-rmo52
and keep going with my dev
13:51 pdurbin
cdsp-rmo52: ok, so you're not completely blocked. good!
13:51 cdsp-rmo52
yep :)
14:04 cdsp-rmo52
pdurbin: I have a question about the dataset import
14:06 cdsp-rmo52
a field like "Subject" is required for a dataset. So, I was wondering: what should I put there ? Given it is not really part of ddi standards, how am I supposed to fill this field ? :S
14:25 donsizemore joined #dataverse
14:45 Jim__
@pdurbin - was looking for info on transferring existing files to s3 after switching to S3 storage as discussed in the config guide.
14:47 Jim__
it looks simple - sync the files and prepend s3://<bucketname>: to the storageidentifiers in the dvobject table, but wanted to know if that was documented or if I'm missing needed steps, etc.
14:49 pameyer joined #dataverse
14:53 pameyer
cdsp-rmo52: "subject" wasn't really meaningful for our installation (everything wouldn've had the same subject). I put some functionality in our fork to auto-select if there was only a single subject in the citation metadata, removed all the other subjects, and hid it
14:54 pameyer
tcoupin: if you're in docker, you definately need siteUrl to publish (like cdsp-rmo mentioned). this didn't used to be the case, and I'm not entirely sure what changed the behavior; but it's tripped me up in the past
14:59 pdurbin
Jim__: the best thing might be for you to email supporthttps://github.com/IQSS/dataverse/issues/4309
15:03 pdurbin
cdsp-rmo52: in addition to what pameyer said about subjects, we could talk about the "N/A" option if you want: http://guides.dataverse.org/en/4.9.2/api/sword.html#new-features-as-of-v1-1
15:06 Jim__ joined #dataverse
15:06 Jim__
Thanks - this looks like step ~7.2 in the checklist mentioned in that issue - I'll ping support for details...
15:08 pdurbin
Jim__: sure, if you keep clicking through issues you'll find a couple google docs. I can link them here if you'd like.
15:09 pdurbin
Jim__: the google doc linked from here seems to be the final cutover day checklist: https://github.com/IQSS/dataverse/issues/4322#issuecomment-367492690
15:28 Jim__
got it - that appears to confirm what I was thinking and gives the queries needed. Thanks!
15:32 pdurbin
sure thing
15:32 pdurbin
pameyer: question about astyle when you have a moment
15:34 pameyer
pdurbin: sure
15:45 pdurbin
pameyer: would you be offended if I replace what you wrote about astyle with "run `mvn checkstyle`"?
15:46 pameyer
depends - does it work better?
15:46 pameyer
;) seriously, no
15:47 pdurbin
cool, I'll take it out. I'll ping you for code review in a bit
15:47 pameyer
I'd thought the checkstyle PR was about checking code, not reformatting it
15:47 pameyer
no problem - I took a quick look when it came in
15:47 pdurbin
well, this will be a new, smaller pull request
15:50 pameyer
gotcha
15:50 pameyer
hopefully checkstyle can handle the "only reformat the code you touched" part
15:51 pdurbin
nope
15:51 pdurbin
well, it'll tell you which lines are in violation
15:54 pameyer
with > 600 of those, that may not be too helpful. but maybe I should wait until you *ask* for review
16:06 pdurbin
pameyer: ok, I clicked "request review"
16:06 pdurbin
Jim__: still there?
16:21 pameyer
pdurbin: reviewed
16:21 pameyer
and ~200 vs > 600
16:29 pdurbin
200 vs 600? not sure what you mean
16:29 pameyer
reported violations
16:30 pdurbin
astyle reports more or fewer?
16:32 pdurbin
"There are 199 errors reported by Checkstyle 6.18 with checkstyle.xml ruleset."
16:32 pdurbin
So I guess you're saying that astyle reports more.
16:35 pameyer
no - when I ran the original PR, I thought its configuration reported ~600 checkstyle violations
16:35 pameyer
I might be remembering wrong, of course
16:36 pdurbin
oh, possibly. the newer pull request only has two checks: no tabs and where to put left curly braces
16:37 pdurbin
the older pull request had addition checks. I'm not even sure what they were all for
16:45 pdurbin
andrewSC: fixed: https://github.com/IQSS/dataverse/issues/4558
16:53 pdurbin
pameyer: this should make you happy: https://github.com/IQSS/dataverse/commit/f3d9b31 :)
16:55 pameyer
pdurbin: I like being able to know when stuff's broken
16:55 pdurbin
me too
16:55 pdurbin
speaking of broken stuff, phoenix is now reporting failed API tests on "develop"
16:56 pdurbin
any interest in running your docker-aio tests?
16:56 pameyer
I saw - I kicked stuff off on my end
16:56 pameyer
didn't see anything broken last night; but being systematic
16:56 pdurbin
awesome. thanks you!
16:56 pdurbin
yeah, no failures on develop as of 8pm last night
16:57 pdurbin
I'm doing another run on phoenix right now. Additional pull requests were merged. Maybe one will magically fix the failures? :) Doubt it.
17:01 pdurbin
pameyer: issue created: https://github.com/IQSS/dataverse/issues/5107
17:06 pameyer
probabably doesn't do me any good to test ddc41f58c1cc21449eb5d1f9a69a13be7915ca8d ; since I forgot to pull develop first :<
17:31 pameyer
I'm wondering if a test handle server exists, and if so if the configs for it have issues
17:32 pdurbin
phoenix is not configured to use a handle server. it uses DataCite
17:32 pameyer
well, I'm not sure why else I'd be seeing key exchange failures in the server log
17:32 pameyer
:doh: other than CA certs, potentially
17:33 pameyer
but that's for after lunch
17:46 donsizemore joined #dataverse
18:22 pameyer
donsizemore: does dataverse-ansible rely in the default ezid credentials?
18:23 donsizemore
@pameyer they're configurable as group_vars: doi: authority: "10.5072" protocol: doi shoulder: "FK2/" provider: EZID
18:23 donsizemore
@pameyer (and could be passed to ansible-playbook as --extra-vars if need be)
18:23 pameyer
well, develop removed the default username/password from the install
18:24 donsizemore
@pameyer should i go ahead and update the role?
18:24 pameyer
probably ;)
18:24 pameyer
might be worth double-checking that you need to first though
18:49 pdurbin
pameyer: Steve and I fixed and closed the API test issue I opened because (per my latest comment), the phoenix server is happy again. If docker-aio is showing failures after you pull the latest from develop, do you mind opening a fresh issue?
18:50 pameyer
pdurbin: as usual
18:50 pdurbin
hmm? :)
18:51 pameyer
don't I usually open issues about broken stuff?
18:51 pdurbin
yes, yes, and thank you! :)
18:51 pameyer
glad you two got it sorted though - I got pulled into other stuff
18:51 pdurbin
actually if you end up making a pull request there's a bit of a doc change we should add
18:52 pdurbin
This is no longer true: "Out of the box, Dataverse is configured with a test username from EZID" http://guides.dataverse.org/en/4.9.2/installation/config.html#doi-username
19:16 pameyer
the "out of the box" issue is the one that I don't have a great idea about how to sort out, based on the comments on the various issues
19:17 pameyer
*might* be able to spin up cheetoh and hook things up that way
19:17 Mahsa joined #dataverse
19:18 Mahsa
Good afternoon, I checked with our IdP and they are providing us with more attributes, although we still are not receiving them.
19:18 Mahsa
I checked the Shibboleth logs and found that the attributes are skipped as they can not be found/mapped
19:19 Mahsa
I also noticed that our IDP are putting the attribute "id" in the "name" tag, and there is no "id" tage in what they are sending
19:20 Mahsa
Now, my question is would setting up the "Name" and "Id"s properly simply solve the issue. My concern is the format of the values that are being sent.
19:20 pdurbin
cheetoh?
19:20 Mahsa
Isn't that alo get validated by Attribute resolver ?
19:22 pdurbin
Mahsa: I'm confused. Are they sending attributes you need but with different names? You need "eppn" but they're sending "myawesomeid" or whatever instead?
19:22 Mahsa
Yes, not that different
19:22 Mahsa
Attribute tags have "Id" and "Name"
19:22 pdurbin
Do you see the attributes on that "Session" page we talked about yesterday?
19:23 Mahsa
No, Shibboleth skips them, because it can not map them
19:23 Mahsa
since the Names do not match
19:23 pdurbin
huh
19:24 Mahsa
what is supposed to be in the "Id" tag , is in the "name" tag
19:24 Mahsa
for the attrbutes
19:24 pdurbin
I'm not sure what "Id" is. It sounds a bit like "eppn" though.
19:25 Mahsa
here is an example
19:25 Mahsa
this is how the IDP is sending an attribute :
19:25 Mahsa
<saml:Attribute Name="cn" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:basic">
19:26 Mahsa
while in the in the attribute map it looks like this:
19:26 Mahsa
<Attribute name="urn:mace:dir:attribute-def:cn" id="cn"/>
19:26 pdurbin
right, that's what I'm seeing in http://guides.dataverse.org/en/4.9.2/_downloads/attribute-map.xml
19:26 Mahsa
so I think Shibboleth can not map them and just skips them
19:27 Mahsa
So, I will ask them to sort this out
19:27 Mahsa
But my question is
19:27 Mahsa
Are the values for these attributes also get validated ?
19:27 pdurbin
Dataverse doesn't need "cn" by the way. Are they sending any attributes that Dataverse needs?
19:27 Mahsa
yes, I sent them a bif list, none can be mapped because of the "naming" issue
19:27 pdurbin
oh
19:28 pdurbin
what do you mean by validated?
19:28 Mahsa
meaning that the values for those attributes should follow a certain format
19:28 Mahsa
?
19:29 pdurbin
Hmm. I'm not sure. Did you see the "diff" attachments on that thread you found? Where Alex got NameId working for eppn?
19:29 Mahsa
No, I will go back and check
19:30 pdurbin
Mahsa: those diffs are linked from https://groups.google.com/d/msg/dataverse-community/PXHKNX6m-eU/eheWD4-3AAAJ
19:31 pdurbin
Hmm. I just looked at them and I'm not sure if they'll help or not. Are you aware of the shib users mailing list?
19:32 Mahsa
Yes, I think the issue here is that our IDP is not using the SAML atributes and it seems they are creating custom ones
19:32 Mahsa
which I dont get why
19:33 Mahsa
mhm,, I will look into it , Thank you pdurbin
19:33 pdurbin
Is your IDP powered by Shibboleth or some other software?
19:33 Mahsa
mhmm, I dont know the answer
19:34 pdurbin
ok, it might be worth asking
19:34 Mahsa
sure, Thanks
19:34 Mahsa
I am sure it is not Shibboleth
19:34 pdurbin
ah, interesting
19:34 pdurbin
well, SAML is supposed to be a standard
19:34 Mahsa
Yes, exactly
19:34 pdurbin
so hopefully we'll get it working :)
19:35 Mahsa
sure, I am aware that another software has used Shibboleth with our idp for SSO
19:35 Mahsa
so this should work as well
19:35 pdurbin
ok, good
19:35 Mahsa
I will check the SAML attributes and see why our IDP is not using standard attributes
19:36 pdurbin
perfect
19:36 Mahsa
thanks again
19:36 pdurbin
sure
19:37 pdurbin
pameyer: I went ahead and made a pull request for the EZID doc change: https://github.com/IQSS/dataverse/pull/5110
22:09 pameyer left #dataverse
