Time
S
Nick
Message
01:20 jri joined #dataverse
04:19 jri joined #dataverse
06:58 jri joined #dataverse
07:54 mbamouni joined #dataverse
08:01 stefankasberger joined #dataverse
08:43 poikilotherm joined #dataverse
11:53 pdurbin
stefankasberger: hi, for now I'm using requests directly to publish datasets: https://github.com/IQSS/dataverse-sample-data/commit/a1b4073 . So I'm not blocked. No rush.
12:32 stefankasberger
@pdurbin: you mean the get_request function in pyDataverse, or the requests library itself?
12:39 pdurbin_m joined #dataverse
12:39 pdurbin_m
stefankasberger: the requests library itself
13:09 donsizemore joined #dataverse
13:10 donsizemore
@pdurbin we found the culprit in the code; akio and gustavo have different ideas on what to do about it
13:10 donsizemore
@pdurbin but gustavo's 4.15-prod-patch7-1 branch reduced the memory consumption dramatically
13:16 pdurbin
donsizemore: I have all sorts of questions. :)
13:17 donsizemore
@pdurbin i do too now that i'm swinging back around this morning
13:18 pdurbin
donsizemore: first, do you have a shell on the Glassfish servers? Is that how you can look at memory consumption?
13:19 donsizemore
@pdurbin i'm hammering on a couple test servers here. i'm lazy and unscientific and restarting glassfish to kill ram, then watching the OS memory usage
13:19 pdurbin
Ok, so you're testing on your own kit. Makes sense. Are you using sar? Something else?
13:19 donsizemore
@pdurbin and yesterday i managed to kill dataverse 4.15 using only curl =)
13:20 donsizemore
@pdurbin this was seat-of-the-pants yesterday afternoon, so i'm watching usage climb with 'free -kw' a shell script
13:21 pdurbin
gotcha
13:21 pdurbin
maybe email the DoS thing to security
13:22 pdurbin
How did you find the culprit? My understanding is that the dataset page was the starting point. Did you just start looking at the git history to see what had been changed? Or did you use a tool to help identify the culprit?
13:23 donsizemore
did you see my PMs?
13:24 pdurbin
Yep. I'm just wondering if you used MAT or whatever.
13:24 pdurbin
Eclipse Memory Analyzer Tool (MAT): http://guides.dataverse.org/en/4.15/developers/tools.html#eclipse-memory-analyzer-tool-mat
13:34 pdurbin
It's been a while since I've used MAT.
13:41 pdurbin
Over at https://github.com/IQSS/dataverse.harvard.edu/issues/18#issuecomment-497799997 I was thinking out loud that I should ask the sysadmins again what tools like like for monitoring Dataverse.
13:42 pdurbin
andrewSC bricas_ icarito[m] jri juancorr mbamouni poikilotherm: any thoughts on monitoring tools? ^^
13:42 poikilotherm
Monitoring of what?
13:42 poikilotherm
Depends on what you are trying to see...
13:43 poikilotherm
Monitoring is a performing art, sometimes related to black magic
13:43 pdurbin
I'm talking about monitoring Dataverse. If you were to rewrite http://guides.dataverse.org/en/4.15/admin/monitoring.html what would you add or remove? donsizemore I meant to mention you too
13:44 poikilotherm
A word or two about logging to an ELK or EFK stack (or similar) may be a good idea
13:45 pdurbin
Right now all that page says is "Are you excited about the ELK stack (Elasticsearch, Logstash, and Kibana)? "
13:45 poikilotherm
Oh does it?
13:45 poikilotherm
Ah fund it
13:45 pdurbin
"Are you excited about the ELK stack (Elasticsearch, Logstash, and Kibana)? The TICK stack (Telegraph InfluxDB Chronograph and Kapacitor)? GoAccess? Prometheus? Graphite? Splunk? Please consider sharing your work with the Dataverse community!"
13:46 poikilotherm
Hehehe, yeah, all the good stuff in there
13:46 pdurbin
That was me just adding a laundry list of tools. I haven't used any of them.
13:46 poikilotherm
ELK/EFK is an emerging standard using OSS without additional cost
13:47 poikilotherm
Obviously Graylog and Splunk are good alternatives if you want to spend money
13:48 poikilotherm
ELF/EFK is very easy to deploy either on classic and container setups
13:49 poikilotherm
Regarding time series you definitly should mention Grafana
13:49 pdurbin
poikilotherm: is it in scope to add some optional monitoring to https://github.com/IQSS/dataverse-kubernetes ?
13:49 jri
pdurbin: For now we're using icinga+nrpe for system monitoring and Grafana to visualize systems metrics and Graylog for the logs analysis (it's like ELK, same thing but different). It's quite basic for now but good enough for us. But I'm actually working on a new installation and I would like to monitor also the Metrics API and Java/Glassfish stuff so I'll probably have much more to say in few weeks.
13:49 pdurbin
donsizemore: or to https://github.com/IQSS/dataverse-ansible ?
13:49 poikilotherm
I am very unsure. Maybe add doc references
13:50 poikilotherm
Yeah, MicroProfile should come in handy for this...
13:50 poikilotherm
(@jri)
13:50 pdurbin
poikilotherm: you're welcome to link to that "monitoring" page for now.
13:50 pdurbin
jri: ah, a new installation. Nice!
13:51 jri
Yes, a more "institutional" one, with eduGain and everything
13:55 poikilotherm
jri you might be interested in https://github.com/IQSS/dataverse/issues/5974
13:55 poikilotherm
It's my current focus in https://github.com/orgs/IQSS/projects/4
13:56 Richard_Valdivia joined #dataverse
13:57 poikilotherm
pdurbin: I'm a bit grumpy... Integration testing in Java EE is a f***g nightmare...
13:57 pdurbin
jri: oh, that reminds me. Please consider creating your own GitHub Project under IQSS for Sciences Po.
13:57 Richard_Valdivia
hello all!!
13:57 pdurbin
Richard_Valdivia: welcome back!
13:57 pdurbin
poikilotherm: you're trying to test auth?
13:58 poikilotherm
Yes, I am
13:58 jri
my 2 cents: I think the monitoring page don't have to be a big "how to" with ELK, Munin or whatever fashion tech we can use; but more "How can I get that type of data", "where it is in dataverse ?", and then everyone can adapt this to the monitoring solutions that they already have in their institution. something like that :/
13:58 Richard_Valdivia
poikilotherm: the word "Integration" is nightmare... kk
13:58 poikilotherm
And it's not going well. I need a running container. But as far I understood youo guys, you don't want Arquillian. So I need other workarounds. Not much on this around...
13:58 pdurbin
jri: that makes sense. Thanks.
13:58 jri
thanks poikilotherm for the lings !
13:59 jri
links*
13:59 poikilotherm
I'm looking into embedding Postgres, using Maven plugin for glassfish and such things
14:00 poikilotherm
As we are not able to run on Payara (yet), this is not getting easier :-(
14:00 pdurbin
poikilotherm: what's the next step toward running on Payara. I'm feeling blocked.
14:00 poikilotherm
I even thought again about trying Payara 4, just as an impression of my frustration :-(
14:00 Richard_Valdivia
pdurbin: Hi!! Flavio told me that he met you at the meeting!! in next days we will work in thats job with metadata!
14:01 jri
pdurbin: In facts, that's what I like to read and need as a sysadmin, but if the reader is not a techy person or a computer scientist it could be hard to understand how to do monitoring without examples, I don't know...
14:02 pdurbin
Richard_Valdivia: yes! Me and Flavio: https://twitter.com/BlueJeansDiva/status/1142177832535318529
14:02 poikilotherm
jri: absolutely. A nice overview where to look at and maybe some people as references, should be enough
14:02 poikilotherm
jri: people interested in more details could ask the guys mentioned for more within the community
14:04 poikilotherm
jri: just earlier I read an interview with "Uncle Bob", one of the masterminds behind the Agile Manifesto about the role of software today. Devs need to write more about what they do, and hopefully people will write some docs about monitoring to help others get started
14:05 poikilotherm
(For reference, the interview is here: https://www.heise.de/developer/artikel/Uncle-Bob-Nichts-geschieht-in-der-heutigen-Gesellschaft-ohne-Software-4442721.html . You might run it through DeepL for translation, it's very interesting)
14:06 pdurbin
Uncle Bob speaks German?
14:06 poikilotherm
I dunno. Most certainly it has been translated...
14:06 poikilotherm
Heise is a big IT publisher here in Germany
14:07 poikilotherm
Uncle Bob in Wikipedia: https://en.wikipedia.org/wiki/Robert_C._Martin
14:09 pdurbin
right, the clean code guy, I've heard him on podcasts
14:09 poikilotherm
Oh he seems to speak German
14:10 poikilotherm
The author responded in a forum thread asking if the interview is available in English that they exchanged the draft in German
14:12 Richard_Valdivia joined #dataverse
14:14 pdurbin
nice
14:16 poikilotherm
pdurbin: do you think it would be ok to go ahead without adding automated tests for the OAuth2 stuff? This is really blocking me.
14:17 pdurbin
poikilotherm: sure! Not adding automated tests for auth hasn't stopped the rest of us. :)
14:18 poikilotherm
I really dislike leaving it as is, but doing it properly involves a lot of painfull stuff like getting on sth. recent.
14:18 poikilotherm
(Maven Glassfish will not work reliably - custom patching... ;-) )
14:20 pdurbin
We do enough patching of Glassfish already. That's why I'd like to get off Glassfish 4.1. I'm feeling pretty blocked on this. Payara 5 doesn't like our JSF.
14:34 pdurbin
But I'd like for the next pull request to be made to help get us off Glassfish 4.1. I'm not sure what that pull request should look like.
14:48 pdurbin
Whoops. I just killed https://demo.dataverse.org by trying to "download all" 45 files. Sorry about that if anyone here was using it. :(
15:38 pdurbin
demo is back up. phew
15:42 pdurbin
andrewSC bjonnh bricas_ donsizemore jri juancorr mbamouni Richard_Valdivia: the community call will start in about 20 minutes: https://dataverse.org/community-calls
16:07 jri joined #dataverse
19:00 pdurbin
donsizemore: oh, an existing installation upgraded and can now be added to https://dataverse.org/metrics
19:11 donsizemore
@pdurbin hoo? hoo! <insert owl gif here>
19:14 pdurbin
heh
19:14 pdurbin
donsizemore: running 4.15 now: https://data.cifor.org
19:14 pdurbin
bah, maybe it's down now
19:15 donsizemore
@pdurbin i'll add them to the config. we can pick it up in the morning
19:17 pdurbin
Sounds good. Are you out Friday like I am?
19:17 donsizemore
i'll be here
19:18 pdurbin
Ok. I should probably write a script to detect when installations have upgraded to 4.9 or higher.
19:18 pdurbin
But I've also asked them to tell us. So maybe they'll just tell us. :)
