Time
S
Nick
Message
08:10 jri joined #dataverse
09:14 jri_ joined #dataverse
09:32 juancorr joined #dataverse
09:43 poikilotherm joined #dataverse
12:42 donsizemore joined #dataverse
12:49 pdurbin
donsizemore: morning! I was thinking... should we go ahead and move https://github.com/IQSS/dataverse-jenkins/pull/20 to QA now and worry about https://github.com/IQSS/dataverse-jenkins/issues/21 in a future pull request?
12:55 donsizemore
@pdurbin i was dealing with doctors and another project yesterday, was about to start on #21 unless you want to collapse it back into #20
12:57 pdurbin
I'm fine with whatever. (And sorry to hear about the doctors.) I'm just thinking that pull request 20 delivers a ton of value (importing the job "just works" now) and I'd be happy to say that at standup this morning while it's in QA. So if it's ok with you, I'll move it now.
12:58 donsizemore
@pdurbin you can merge it if you like.
12:59 donsizemore
#21 will add in the aws bit (which i stupidly forgot) and maybe #22 can document the updated job
13:02 pdurbin
Merging is probably a better idea. Done. Thanks!
13:12 donsizemore
@pdurbin Q: do you want AWS output to default to JSON ?
13:59 pdurbin
donsizemore: hmm. I'm not sure. What format is the output in now?
14:02 pdurbin
andrewSC bricas donsizemore jri_ JonathanNeal larsks pmauduit poikilotherm: in three hours during the community call we will be discussing upgrading from Glassfish 4.1 to something else. The candidates are Payara, Glassfish 5, TomEE, Wildfly, and Open Liberty. Can you please add anything that's important to you to this spreadsheet?
14:02 pdurbin
https://docs.google.com/spreadsheets/d/1TVVErG_zFC6k4Fnjz8rJ9VJxMD_xGJSk7OwVjQOLXaQ/edit?usp=sharing
14:03 pdurbin
You are also welcome to comment on this "upgrade from glassfish 4.1" issue: https://github.com/IQSS/dataverse/issues/6230
15:21 xarthisius
pdurbin: ICYMI https://github.com/whole-tale/whole-tale/issues/53
15:22 pdurbin
xarthisius: thanks!! So there *was* an issue already. :)
15:50 xarthisius
pdurbin: I read meeting notes related to capsule/tale handling in Dataverse. What you might be intrested in context of "viewing"/"displaying" a Tale is its manifest
15:50 xarthisius
e.g. curl -s https://girder.wholetale.org/api/v1/tale/5cfd57fca18691e5d1feeda6/manifest?expandFolders=false | jq
15:51 xarthisius
of course it's also a part of the Tale itself so you'd have that file "locally"
15:54 xarthisius
among other things: it carries information about other datasets from DV that the Tale is using, which is the whole point of our implementation
16:10 pdurbin
xarthisius: sorry, I'm on a call with Renku, which is highly related! Please see https://github.com/SwissDataScienceCenter/renku-python/issues/668
16:13 hblancoca joined #dataverse
16:13 hblancoca
Failed to add a new style. Error: Error code (500) from GeoServer: :Error persisting LayerInfoImpl[leishmania_qb, resource:FeatureTypeInfoImpl[leishmania_qb]] to /mnt/sdp/opt/geonode/data/workspaces/geonode/wm_201910/leishmania_qb/layer.xml
16:14 hblancoca
does anyone know if there are problems with the geoconnect server ?
16:14 hblancoca
today we cant see map
16:14 hblancoca
:(
17:02 pdurbin
The community call is starting in a couple minutes. Call in information is at https://dataverse.org/community-calls
17:03 pdurbin
hblancoca: you're welcome to join the call! I'm on it so I can't talk about geoconnect right now.
18:00 poikilotherm
Hooray. Finally I can do a profiling
18:25 jri joined #dataverse
18:43 pdurbin
poikilotherm: oh?
18:44 pdurbin
hblancoca: back from lunch and many meetings. I spoke with a geoconnect/worldmap developer and he said worldmap may have been down and that your geoconnect installation is probably fine.
18:48 poikilotherm
pdurbin: https://github.com/IQSS/dataverse/issues/6052#issuecomment-549963700
18:48 poikilotherm
I'm done for today.
18:48 poikilotherm
It's 20:00 over here and need to get some other stuff done, too :-D
18:53 pdurbin
poikilotherm: thanks! Next it would be great to compare develop to Jayanthy's branch.
18:53 poikilotherm
Nope.
18:53 pdurbin
xarthisius: why isn't Whole Tale mentioned at https://www.nature.com/articles/d41586-019-03366-x ?!?
18:53 pdurbin
poikilotherm: why not? :)
18:54 poikilotherm
You can simply take a look at the profile I made without language dir
18:54 poikilotherm
That way the default bundle is used
18:54 poikilotherm
Ok to be fair: Jayanthy provided a workaround.
18:55 poikilotherm
But IMHO it's bad, because you loose the freedom to place the files at a place you want it
18:55 poikilotherm
This is bad for my docker stuff for example
18:55 pdurbin
poikilotherm: sorry, I'm doing many things at once. Are you saying you've measured a difference or not? And if so, what is that difference?
18:55 poikilotherm
That workaround BREAKS the feature completly
18:57 poikilotherm
The difference is: loading from classpath is fast
18:58 pdurbin
How was Java EE designed? To load from the classpath?
19:00 poikilotherm
No no no. This loading from a path is totally fine and standard
19:00 poikilotherm
This is a caching issue
19:00 pdurbin
What I'm wondering is if we are doing something weird in Dataverse in terms of how we internationalize it.
19:00 poikilotherm
The bundles are not cached, but re-read. As the hit counts are at 45, I guess they are loaded for each facet again and again
19:00 poikilotherm
Nope
19:00 poikilotherm
YOu aren't
19:00 pdurbin
phew
19:01 pdurbin
so we need more caching
19:01 poikilotherm
You are totally inline with the standards and using it correctly
19:02 poikilotherm
My wild guess (bit educated): not all bundle strings are loaded during startup.
19:02 poikilotherm
You will crash the app when the props cannot be found while having the lang dir feature configured
19:02 poikilotherm
It just does not start.
19:02 poikilotherm
(Hit that myself)
19:02 pdurbin
hmm. ok
19:02 poikilotherm
So the basic bundles are loaded and cached
19:03 poikilotherm
And ONLY when the search facets get their localization, those are missing
19:03 poikilotherm
Because when accessing a dataset this behaviour does NOT happen
19:03 poikilotherm
I only see this in the call graphs when SearchIncludeFragment.search() is involved
19:04 poikilotherm
Guess continued: it gathers all the fields (seen in the call graph) and gets the localized strings
19:04 poikilotherm
And they seem not to be cachable in that thread, for whatever reason
19:05 poikilotherm
No idea
19:05 poikilotherm
So my proposal would be: test what happens if we PRELOAD all the bundles on startup
19:05 pdurbin
Sounds good. :) Is that easy?
19:06 poikilotherm
Just breaking the feature with a workaround is... well... IMHO a bad idea.
19:06 poikilotherm
I have not looked into it yet. Just wild guesses. Hoping for comments on my findings so far.
19:06 poikilotherm
And it's really getting late
19:07 pdurbin
I'll ask Jayanthy to read though the chatter here. Thanks!
19:07 poikilotherm
Perfect. I do hope my files and screenshots help
19:08 pdurbin
I'm not sure what to do with the files but the chatter here helps me. So thanks! I left a comment.
19:09 poikilotherm
You can load them in VisualVM
19:09 poikilotherm
And have a look at the cpu graphs
19:10 pdurbin
ok, thanks
19:10 pdurbin
I should probably install VisualVM. I think it's built into Netbeans.
19:15 poikilotherm
Okidoki.
19:15 poikilotherm
Also left a comment on the PR https://github.com/IQSS/dataverse/pull/6332#issuecomment-549974852
19:16 poikilotherm
I. Need. To. Head. Home. Trouble. Ahead. With. Worktimes.
19:16 poikilotherm
Read you all tomorrow
19:27 dataverse-user joined #dataverse
20:07 pdurbin
xarthisius: I'm in another meeting but I do plan to circle back soon read what you wrote earlier.
20:49 poikilotherm joined #dataverse
20:57 poikilotherm
pdurbin: I just came across http://github.com/poikilotherm/dataverse/blob/b46243d5cb8ebc89fb218630b900a6609410a491/src/main/java/edu/harvard/iq/dataverse/search/SearchServiceBean.java#L124-L783
20:57 poikilotherm
A very interesting piece of Dataverse code
21:10 pdurbin
poikilotherm: what part? :)
21:12 poikilotherm
All of those beautiful 659 lines
21:13 pdurbin
xarthisius: I think I see you what you mean. You reference the dataset like this: "schema:isPartOf": "doi:10.5065/D6862DM8"
21:14 pdurbin
poikilotherm: I wrote a lot of those lines. A long time ago. Sins of the past. :(
21:15 poikilotherm
git annotate already revealed that ;-)
21:15 pdurbin
You should rewrite it and make it awesome. :)
21:16 poikilotherm
No. We should create an issue...
21:21 pdurbin
You're right. I forgot our process. :)
21:21 poikilotherm
;-)
21:22 poikilotherm
I bet it's gonna be tagged as "Too large to estimate"
21:31 pdurbin
:)
21:31 pdurbin
wouldn't be the first time :)
22:48 poikilotherm joined #dataverse
