Time
S
Nick
Message
07:12 dataverse-user joined #dataverse
07:12 nightowl313 joined #dataverse
07:13 nightowl313
anyone have an idea why, when I try to preview spreadsheet files I get an error "Unable to retrieve metadata.Please try again. The most common issue is that your login has timed out. If the problem persists, please contact the support team of this data repository. Please include any status code included at the end of this message: error"?
07:14 nightowl313
and when I try to Explore them, I just get the blue bar at the top spinning like it's trying to load the file, but never does
07:17 nightowl313
i installed using dataverse-ansible with previewer and external tools enabled
08:16 jri joined #dataverse
08:43 stefankasberger joined #dataverse
08:55 Benjamin_Peuch joined #dataverse
10:16 patrickv joined #dataverse
10:17 patrickv
Hi. I'm testing harvesting with two tests servers. I have a problem.
10:18 patrickv
I activated the harvesting server, defined a set based on doi.
10:19 patrickv
When I connect to URL http://myserver/oai I get an internal server error
10:20 patrickv
in the glassfish logs, I have: Allocate exception for servlet OAIServlet:java.net.MalformedURLException: no protocol: master/oai
10:21 patrickv
In the harvesting client, I used the url : http://myserver/oai . Is it right ?
10:52 poikilotherm
Hi patrickv
10:53 poikilotherm
Have you tried accessing OAI on both servers like this: https://demo.dataverse.org/oai?verb=Identify
10:54 poikilotherm
That should give you a proper response. You can compare with the output of the demo instance to be sure things are ok
10:54 poikilotherm
If that simple query works, OAI on your servers should be working in a general manner. There still might be other hidden problems.
10:55 poikilotherm
What Dataverse version are you using?
11:55 patrickv
I use version 4.14
11:56 patrickv
On the harvesting server, I defined the set dsPersistentId:"doi:10.5072/"
11:56 patrickv
It finds one record : 1 dataset (1 record exported, 0 marked as deleted)
11:59 poikilotherm
Ui. Current release is 4.19 and IIRC there had been some bugfixes in the meantime...
11:59 poikilotherm
pdurbin is most likely with us in a short while
11:59 poikilotherm
He will dig up those issues and related stuff in no time
12:00 pdurbin
Yeah, 4.14 is somewhat old at this point. I wonder if the exception is thrown in 4.19.
12:00 pdurbin
patrickv: do you feel like creating an issue about it?
12:00 patrickv
the url http://myserver/oai?dsPersistentId:10.5072 causes the internal server error
12:04 pdurbin
If I go to https://dev2.dataverse.org/oai?dsPersistentId:10.5072 I see badVerb
12:21 pdurbin
I wonder if we should document OAI-PMH in the Dataverse API Guide. It sure seems like an API to me.
12:23 donsizemore joined #dataverse
12:26 poikilotherm
https://www.openarchives.org/OAI/2.0/guidelines.htm ?
12:26 poikilotherm
https://support.datacite.org/docs/datacite-oai-pmh
12:27 pdurbin
Yeah, I like those docs from DataCite.
12:31 pdurbin
poikilotherm donsizemore: have you seen https://www.slideshare.net/vty/the-world-of-docker-and-kubernetes via https://twitter.com/4tykhonov/status/1227199925278199808 ?
12:32 poikilotherm
pdurbin: I'm about to create the first integration guide for the cloud & container guide, describing auth providers :-)
12:32 poikilotherm
https://github.com/IQSS/dataverse-kubernetes/issues/166#issuecomment-584593818
12:33 donsizemore
@pdurbin I have not but I'll read the deck once I get more coffee!
12:33 pdurbin
donsizemore: great! Also, I was wondering if you could confirm if there's a shib bug or not. This: https://github.com/IQSS/dataverse/issues/6630#issuecomment-584585654
12:34 pdurbin
poikilotherm: awesome. Go, go, go!
12:44 pdurbin
poikilotherm: on slide 20, "UKDA (UK)" ... I'm thinking about SSI.
12:46 poikilotherm
Oh nice idea!
12:46 patrickv joined #dataverse
12:47 pdurbin
donsizemore: iRODS on slide 48
12:47 pdurbin
poikilotherm: maybe Slava can introduce you to someone there?
12:48 pdurbin
Yeah, these slides are awesome. Good job, Slava.
12:49 pdurbin
If I go to https://dev2.dataverse.org/oai?dsPersistentId:10.5072 I see badVerb, patrickv
12:50 pdurbin
poikilotherm patrickv: check this out. Dorothea is contributing to the map I showed you: https://github.com/IQSS/dataverse-installations/commits?author=doigl :)
12:51 poikilotherm
Finally :-D
12:52 poikilotherm
pdurbin: what is your suggestion where to drop the integrations section in the cloud guide?
12:52 poikilotherm
Should it live next to the others as a main section
12:52 poikilotherm
Or should it be a part of "Day 2"?
12:53 poikilotherm
(or Day1)
12:53 pdurbin
maybe Day 3 :)
12:53 pdurbin
With Day 3 the message is... get the basic working... then get fancy!
12:53 pdurbin
basics*
12:54 pdurbin
fancy! https://pusheen.com/1072-2/
12:54 poikilotherm
Oh that a nice idea
12:54 poikilotherm
I like it
12:56 poikilotherm
LOL
12:58 donsizemore
@pdurbin dataverse-test is only running 4.16, but: dvntest=> select lastlogintime from authenticateduser where email='dls
12:59 dataverse-user joined #dataverse
12:59 pdurbin
d'oh!
12:59 patrickv joined #dataverse
12:59 donsizemore
i can upgrade it to 4.19 if you like
12:59 pdurbin
Meh, leaving a comment would be enough, if you don't mind.
12:59 patrickv
whatever I put after https://myserver/oai? I get an internal server error
13:00 patrickv
I get the same from our prod server... hummm...
13:00 pdurbin
patrickv: what about this verb? https://demo.dataverse.org/oai?verb=Identify
13:00 patrickv
it works
13:00 pdurbin
good
13:01 pdurbin
You always need a verb.
13:01 pdurbin
https://demo.dataverse.org/oai?verb=ListMetadataFormats
13:01 pdurbin
etc
13:02 donsizemore
@patrickv you don't get XML saying "badVerb"?
13:02 patrickv
If I do the same for our server https://dataverse.uclouvain.be I get internal server
13:03 donsizemore
@patrickv what's in server.log?
13:04 patrickv
StandardWrapperValve[OAIServlet]: Allocate exception for servlet OAIServlet java.net.MalformedURLException: no protocol: dataverse.uclouvain.be/oai
13:06 donsizemore
@patrickv 500 error or XML badVerb, the /oai location is merely the base
13:06 pdurbin
andrewSC bjonnh bricas donsizemore jri juancorr patrickv pmauduit poikilotherm stefankasberger yoh: the community call will start in about 2 hours. The Zoom link and notes doc can be found at https://dataverse.org/community-calls
13:07 donsizemore
@patrickv a number of harvesting improvements were included in 4.16 https://github.com/IQSS/dataverse/issues?q=is%3Aclosed+milestone%3A4.16
13:07 patrickv
500 internal error
13:09 donsizemore
@patrickv and I wouldn't want to run 4.14 or 4.15 in production. 4.15.1 corrects a resource leak, and in your case 4.16 offers harvesting improvements
13:11 poikilotherm
patrickv donsizemore pdurbin this error sounds like our good old friend dataverse.fqdn not adding a protocol to the URL ...
13:11 patrickv
OK but it should work. I suspect a problem of configuration but don't where to search for
13:12 pdurbin
patrickv: can you please look for dataverse.siteUrl in domain.xml?
13:13 poikilotherm
pdurbin: https://github.com/IQSS/dataverse/blob/57a392d90dac5d338ccdcd06d176a032f0c89e04/src/main/java/edu/harvard/iq/dataverse/harvest/server/web/servlet/OAIServlet.java#L184
13:14 patrickv
<jvm-options>-Ddataverse.siteUrl=dataverse.uclouvain.be</jvm-options>
13:14 donsizemore
@poikilotherm is correct
13:15 pdurbin
patrickv: ah. You need to add "https://" to that
13:17 poikilotherm
BTW this code hasn't been touched since 4.14
13:17 pdurbin
patrickv: to make it a full URL . And then restart Glassfish.
13:17 poikilotherm
And there should be a validation for this setting
13:17 pdurbin
poikilotherm: I trust whatever donsizemore says in terms of which releases are good. :)
13:18 patrickv56 joined #dataverse
13:18 poikilotherm
The whole thing in https://github.com/IQSS/dataverse/blob/1a2bca6a437fee3f8198242f3d4d557dcc04b01e/src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java#L326 hasn't been touched in a whie
13:19 poikilotherm
But like I said - the siteUrl should be validated on startup so such errors get caught early
13:19 pdurbin
patrickv56: if you are getting disconnected from this chat room a lot, you can also connect using http://webchat.freenode.net/?channels=#dataverse
13:19 pdurbin
poikilotherm: and we should get rid of dataverse.fqdn
13:19 donsizemore
@poikilotherm I'd still want to update to 4.16
13:20 poikilotherm
pdurbin: I second that!
13:20 poikilotherm
donsizemore: I second that too!
13:20 pdurbin
poikilotherm: please feel free to create an issue.
13:21 patrickv56
seems good on my test server
13:21 poikilotherm
YOu sure there isn't one already?
13:22 patrickv joined #dataverse
13:25 poikilotherm
donsizemore: you second getting rid of dataverse.fqdn and validate dataverse.siteurl?
13:26 donsizemore
@poikilotherm i get bit often enough by it, how could i not?
13:29 poikilotherm
pdurbin donsizemore https://github.com/IQSS/dataverse/issues/6636
13:30 patrickv
The config is now ok on production server... I will plan an upgrade as soon as possible :-)
13:30 patrickv
Thank you very much !
13:31 poikilotherm
You're most welcome :-)
13:31 MrK joined #dataverse
13:31 poikilotherm
YOu might consider commenting on the issue I posted above that you just got bitten by this
13:31 patrickv
OK !
13:31 pdurbin
poikilotherm: great issue! Thanks!
13:32 pdurbin
for siteUrl, see also https://github.com/IQSS/dataverse/pull/4887
13:34 pdurbin
MrK: community call in an hour and a half if you're interested: https://dataverse.org/community-calls
13:40 MrK
pdurbin: thx for info.
13:47 pkiraly joined #dataverse
13:52 donsizemore
@pdurbin jacoco Q?
13:59 pdurbin
MrK: sure
13:59 pdurbin
donsizemore: yep?
14:00 pkiraly
pdurbin: you said yesterday, that in the PR there are too many commits. What would be the ideal size or unit?
14:00 pdurbin
pkiraly: sorry, it's not the number of commits. That doesn't matter. It's how long the "diff" is.
14:02 pdurbin
Shorter is better, in general. :)
14:02 pdurbin
Smaller diffs make code review easier.
14:02 pkiraly
pdurbin: ok, but it means I'll create multiple PRs
14:02 pdurbin
That's fine. Multiple PRs are fine.
14:03 poikilotherm
pdurbin: I think it's always a tradeoff. You guys accepted those rather large PRs from Jim, too
14:03 pdurbin
Part of the problem is that the "native" API page is very long. We should probably split it up into multiple pages at some point.
14:03 pdurbin
poikilotherm: sure but they take forever to review
14:03 pdurbin
And yes, sometimes long pull requests are unavoidable.
14:04 poikilotherm
Yeah I noticed that, too ;-)
14:05 juancorr joined #dataverse
14:14 poikilotherm
pdurbin: I'll try to join community meeting, but I have no mic around. So listen only for me.
14:29 donsizemore
@pdurbin so, jacoco.
14:38 pdurbin
poikilotherm: we won't ask you to say anything :)
14:38 pdurbin
donsizemore: yes, we have 20 minutes until the call. Go. :)
14:40 pdurbin
pkiraly: https://github.com/IQSS/dataverse/pull/6637 looks good and I moved it over but DOIs are also supported, right?
14:44 pkiraly
pdurbin: I haven't checked if DOIs are supported or not. Now I just replaced variables
14:45 pdurbin
pkiraly: ok, that's fine. I assume they work.
14:45 pkiraly
There are some calls with ID and DOI examples, but those are the minority. In most of calls it is either ID or DOI mentioned
14:45 pdurbin
yeah
14:46 pdurbin
My guess is that most people would rather use DOIs than database IDs.
14:46 pdurbin
when calling into APIs, I mean
14:46 pdurbin
DOIs are much more visible in the UI and in URLs.
14:50 pkiraly
100% agree. IDs are practically invisible, the only way to get them is via API calls
14:55 nightowl313 joined #dataverse
15:02 pdurbin
Here's the doc for the community call. Starting soon: https://docs.google.com/document/d/13Huxestxl_Mt18AgMsc34dS9lqgrUPx6uHbuXAOBSWw/edit?usp=sharing
15:42 poikilotherm
pdurbin please let me know what you thnk https://dataverse-k8s.readthedocs.io/en/166-auth-persistance/day3/auth.html
15:43 pdurbin
nice diagram
15:44 pdurbin
I think it's good that you link to the main guides. I'm having a little trouble following this. Maybe it's because I didn't write it. :)
15:47 poikilotherm
Yeah. That's why I'm asking.
15:47 poikilotherm
It's one thing to write it
15:48 poikilotherm
But another to write it so others can understand :-D
15:48 poikilotherm
I'm still thinking about creating ASCII Cinema or similar
15:49 poikilotherm
Well, gotta run
15:49 poikilotherm
Cu guys
17:56 sj joined #dataverse
17:56 _sj_ joined #dataverse
17:56 _sj_
\wave
17:56 pdurbin
:)
17:57 pdurbin
_sj_: nice to see you here!
17:59 _sj_
pdurbin thanks for the pointers to freya
17:59 _sj_
(:
17:59 pdurbin
Sure! The bummer about Freya is that the grant is ending. That's my understanding anyway.
18:07 pdurbin
But it's neat technology. There was a talk about it at last summer's Dataverse community meeting: https://osf.io/kmhfa/
18:41 sj joined #dataverse
19:15 donsizemore joined #dataverse
