IQSS logo

IRC log for #dataverse, 2019-11-05

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
08:10 jri joined #dataverse
09:14 jri_ joined #dataverse
09:32 juancorr joined #dataverse
09:43 poikilotherm joined #dataverse
12:42 donsizemore joined #dataverse
12:49 pdurbin donsizemore: morning! I was thinking... should we go ahead and move https://github.com/IQSS/dataverse-jenkins/pull/20 to QA now and worry about https://github.com/IQSS/dataverse-jenkins/issues/21 in a future pull request?
12:55 donsizemore @pdurbin i was dealing with doctors and another project yesterday, was about to start on #21 unless you want to collapse it back into #20
12:57 pdurbin I'm fine with whatever. (And sorry to hear about the doctors.) I'm just thinking that pull request 20 delivers a ton of value (importing the job "just works" now) and I'd be happy to say that at standup this morning while it's in QA. So if it's ok with you, I'll move it now.
12:58 donsizemore @pdurbin you can merge it if you like.
12:59 donsizemore #21 will add in the aws bit (which i stupidly forgot) and maybe #22 can document the updated job
13:02 pdurbin Merging is probably a better idea. Done. Thanks!
13:12 donsizemore @pdurbin Q: do you want AWS output to default to JSON?
13:59 pdurbin donsizemore: hmm. I'm not sure. What format is the output in now?
14:02 pdurbin andrewSC bricas donsizemore jri_ JonathanNeal larsks pmauduit poikilotherm: in three hours during the community call we will be discussing upgrading from Glassfish 4.1 to something else. The candidates are Payara, Glassfish 5, TomEE, Wildfly, and Open Liberty. Can you please add anything that's important to you to this spreadsheet?
14:02 pdurbin https://docs.google.com/spreadsheets/d/1TVVErG_zFC6k4Fnjz8rJ9VJxMD_xGJSk7OwVjQOLXaQ/edit?usp=sharing
14:03 pdurbin You are also welcome to comment on this "upgrade from glassfish 4.1" issue: https://github.com/IQSS/dataverse/issues/6230
15:21 xarthisius pdurbin: ICYMI https://github.com/whole-tale/whole-tale/issues/53
15:22 pdurbin xarthisius: thanks!! So there *was* an issue already. :)
15:50 xarthisius pdurbin: I read meeting notes related to capsule/tale handling in Dataverse. What you might be intrested in context of "viewing"/"displaying" a Tale is its manifest
15:50 xarthisius e.g. curl -s https://girder.wholetale.org/api/v1/tale/5cfd57fca18691e5d1feeda6/manifest?expandFolders=false | jq
15:51 xarthisius of course it's also a part of the Tale itself so you'd have that file "locally"
15:54 xarthisius among other things: it carries information about other datasets from DV that the Tale is using, which is the whole point of our implementation
16:10 pdurbin xarthisius: sorry, I'm on a call with Renku, which is highly related! Please see https://github.com/SwissDataScienceCenter/renku-python/issues/668
16:13 hblancoca joined #dataverse
16:13 hblancoca Failed to add a new style. Error: Error code (500) from GeoServer: :Error persisting LayerInfoImpl[leishmania_qb, resource:FeatureTypeInfoImpl[leishmania_qb]] to /mnt/sdp/opt/geonode/data/workspaces/ge​onode/wm_201910/leishmania_qb/layer.xml
16:14 hblancoca does anyone know if there are problems with the geoconnect server ?
16:14 hblancoca today we cant see map
16:14 hblancoca :(
17:02 pdurbin The community call is starting in a couple minutes. Call in information is at https://dataverse.org/community-calls
17:03 pdurbin hblancoca: you're welcome to join the call! I'm on it so I can't talk about geoconnect right now.
18:00 poikilotherm Hooray. Finally I can do a profiling
18:25 jri joined #dataverse
18:43 pdurbin poikilotherm: oh?
18:44 pdurbin hblancoca: back from lunch and many meetings. I spoke with a geoconnect/worldmap developer and he said worldmap may have been down and that your geoconnect installation is probably fine.
18:48 poikilotherm pdurbin: https://github.com/IQSS/dataverse/issues/6052#issuecomment-549963700
18:48 poikilotherm I'm done for today.
18:48 poikilotherm It's 20:00 over here and need to get some other stuff done, too :-D
18:53 pdurbin poikilotherm: thanks! Next it would be great to compare develop to Jayanthy's branch.
18:53 poikilotherm Nope.
18:53 pdurbin xarthisius: why isn't Whole Tale mentioned at https://www.nature.com/articles/d41586-019-03366-x ?!?
18:53 pdurbin poikilotherm: why not? :)
18:54 poikilotherm You can simply take a look at the profile I made without language dir
18:54 poikilotherm That way the default bundle is used
18:54 poikilotherm Ok to be fair: Jayanthy provided a workaround.
18:55 poikilotherm But IMHO it's bad, because you loose the freedom to place the files at a place you want it
18:55 poikilotherm This is bad for my docker stuff for example
18:55 pdurbin poikilotherm: sorry, I'm doing many things at once. Are you saying you've measured a difference or not? And if so, what is that difference?
18:55 poikilotherm That workaround BREAKS the feature completly
18:57 poikilotherm The difference is: loading from classpath is fast
18:58 pdurbin How was Java EE designed? To load from the classpath?
19:00 poikilotherm No no no. This loading from a path is totally fine and standard
19:00 poikilotherm This is a caching issue
19:00 pdurbin What I'm wondering is if we are doing something weird in Dataverse in terms of how we internationalize it.
19:00 poikilotherm The bundles are not cached, but re-read. As the hit counts are at 45, I guess they are loaded for each facet again and again
19:00 poikilotherm Nope
19:00 poikilotherm YOu aren't
19:00 pdurbin phew
19:01 pdurbin so we need more caching
19:01 poikilotherm You are totally inline with the standards and using it correctly
19:02 poikilotherm My wild guess (bit educated): not all bundle strings are loaded during startup.
19:02 poikilotherm You will crash the app when the props cannot be found while having the lang dir feature configured
19:02 poikilotherm It just does not start.
19:02 poikilotherm (Hit that myself)
19:02 pdurbin hmm. ok
19:02 poikilotherm So the basic bundles are loaded and cached
19:03 poikilotherm And ONLY when the search facets get their localization, those are missing
19:03 poikilotherm Because when accessing a dataset this behaviour does NOT happen
19:03 poikilotherm I only see this in the call graphs when SearchIncludeFragment.search() is involved
19:04 poikilotherm Guess continued: it gathers all the fields (seen in the call graph) and gets the localized strings
19:04 poikilotherm And they seem not to be cachable in that thread, for whatever reason
19:05 poikilotherm No idea
19:05 poikilotherm So my proposal would be: test what happens if we PRELOAD all the bundles on startup
19:05 pdurbin Sounds good. :) Is that easy?
19:06 poikilotherm Just breaking the feature with a workaround is... well... IMHO a bad idea.
19:06 poikilotherm I have not looked into it yet. Just wild guesses. Hoping for comments on my findings so far.
19:06 poikilotherm And it's really getting late
19:07 pdurbin I'll ask Jayanthy to read though the chatter here. Thanks!
19:07 poikilotherm Perfect. I do hope my files and screenshots help
19:08 pdurbin I'm not sure what to do with the files but the chatter here helps me. So thanks! I left a comment.
19:09 poikilotherm You can load them in VisualVM
19:09 poikilotherm And have a look at the cpu graphs
19:10 pdurbin ok, thanks
19:10 pdurbin I should probably install VisualVM. I think it's built into Netbeans.
19:15 poikilotherm Okidoki.
19:15 poikilotherm Also left a comment on the PR https://github.com/IQSS/dataverse/pull/6332#issuecomment-549974852
19:16 poikilotherm I. Need. To. Head. Home. Trouble. Ahead. With. Worktimes.
19:16 poikilotherm Read you all tomorrow
19:27 dataverse-user joined #dataverse
20:07 pdurbin xarthisius: I'm in another meeting but I do plan to circle back soon read what you wrote earlier.
20:49 poikilotherm joined #dataverse
20:57 poikilotherm pdurbin: I just came across http://github.com/poikilotherm/dataverse/blob/b46243d5cb8ebc89fb218630b900a6609410a491/src/main/java/edu/harvard/iq/dataverse/search/SearchServiceBean.java#L124-L783
20:57 poikilotherm A very interesting piece of Dataverse code
21:10 pdurbin poikilotherm: what part? :)
21:12 poikilotherm All of those beautiful 659 lines
21:13 pdurbin xarthisius: I think I see you what you mean. You reference the dataset like this: "schema:isPartOf": "doi:10.5065/D6862DM8"
21:14 pdurbin poikilotherm: I wrote a lot of those lines. A long time ago. Sins of the past. :(
21:15 poikilotherm git annotate already revealed that ;-)
21:15 pdurbin You should rewrite it and make it awesome. :)
21:16 poikilotherm No. We should create an issue...
21:21 pdurbin You're right. I forgot our process. :)
21:21 poikilotherm ;-)
21:22 poikilotherm I bet it's gonna be tagged as "Too large to estimate"
21:31 pdurbin :)
21:31 pdurbin wouldn't be the first time :)
22:48 poikilotherm joined #dataverse

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.