IQSS logo

IRC log for #dataverse, 2019-08-06

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
08:06 stefankasberger joined #dataverse
10:56 pdurbin joined #dataverse
11:56 donsizemore joined #dataverse
11:59 donsizemore @pdurbin Dear Leader!
12:19 pdurbin_m joined #dataverse
12:19 pdurbin_m donsizemore: talk to me
12:20 donsizemore @pdurbin welcoming you back
13:03 pdurbin donsizemore: thanks! I highly recommend Banff, Jasper, etc. Great hiking. Cool in the summer.
13:11 donsizemore @pdurbin at this point i'd settle for a day by a pool
13:12 pdurbin oh, we did some hot springs too, great after a big hike
13:19 donsizemore i'm insanely jealous!
13:30 pdurbin donsizemore: should I take a look at your todo column? I've lost track or where we are with everything. :)
13:31 donsizemore @pdurbin i think heppler should be going, he was going to try his sampledatabranch
13:31 pdurbin ah, right, I saw you added the ability to specify the branch, thanks!
13:31 donsizemore @pdurbin also, we added a jenkins job: https://jenkins.dataverse.org/job/IQSS-Dataverse-PullRequest/ (which I think I've corrected for round two which will happen on the next PR submission)
13:32 pdurbin Sounds useful. How much testing does that job do?
13:32 donsizemore right now it's at parity with https://jenkins.dataverse.org/job/IQSS-dataverse-develop/
13:32 donsizemore Gustavo wanted tests against PRs as they came in
13:33 pdurbin Sure but what kinds of tests does he want?
13:36 donsizemore I think optimally the API test suite as well but on that box...
13:37 pdurbin Right, we all want the API test suite. Are we blocked on that?
13:39 donsizemore eh, i just haven't fired it off on that box in a while
13:41 pdurbin Ok. What would help us move forward on getting the API test suite running? Just start firing it off again and remind ourselves what doesn't work?
13:47 donsizemore something knocks glassfish over and it starts returning 500s. despite the same tests on the same code succeeding on your box, which has identical virtual hardware specs
13:48 donsizemore of course, it's been a month, so let me kick it off again
13:52 pdurbin The same thing happens on Glassfish running directly on my Mac. I can't run the full test suite because Glassfish dies in the middle. Strangely, Glassfish doesn't fall over when I run the test suite against Glassfish running within Docker on my Mac.
14:04 donsizemore well, this might do it: Internal Exception: org.postgresql.util.PSQLException: ERROR: deadlock detected
14:05 pdurbin yuck
14:05 pdurbin donsizemore: heads up that the API test suite recently started failing on phoenix. I just opened this issue a couple hours ago: https://github.com/IQSS/dataverse/issues/6073
14:06 donsizemore yes, i saw that
14:07 pdurbin I could ssh into the phoenix server and see if deadlocks are server.log.
14:09 donsizemore [ERROR] Tests run: 97, Failures: 11, Errors: 0, Skipped: 4. I hit FilesIT.testForceReplaceAndUpdate:631 XML path data.files[0].label doesn't match.
14:09 donsizemore after that call everything comes back 500
14:10 pdurbin interesting, on a related note I'm seeing this in the server.log for phoenix: Error processing /api/v1/files/84/replace: javax.ejb.EJBException
14:11 donsizemore @pdurbin i ticked off a number of issues and submitted a couple small PRs i'll let you review at your leisure; in general i still owe you a few closed tickets
14:12 pdurbin Ok, I have 53 unread emails having to do with GitHub issue updates so it may take me a while to find them. :)
14:13 pdurbin Let me go through your column quick: https://github.com/orgs/IQSS/projects/3#column-5977566
14:14 pdurbin Any update on Munin?
14:16 donsizemore for me the postgres deadlock happens right after edu.harvard.iq.dataverse.api.UtilIT createRandomUser
14:16 donsizemore Munin: https://github.com/IQSS/dataverse-ansible/commit/6902d576ee318a6fba09173a7f25c7b610dc28cf
14:17 donsizemore Prometheus: https://github.com/IQSS/dataverse-ansible/commit/5346f05dadf007ea79c1af1766bfa888b16af116
14:17 pdurbin Prometheus! Very cool.
14:19 pdurbin Should I spin up an ec2 instance with prometheus and you can show me around? Is it anything like Munin? I love how Munin "just works" and starts creating a dashboard of graphs within minutes.
14:20 donsizemore you're welcome to spin up an instance with both. i got the basic framework in place and am happy to work on customization
14:20 pdurbin Cool. Is there a dashboard?
14:20 pdurbin A dashboard of graphs like this I mean: http://munin.greptilian.com/greptilian.com/server3.greptilian.com/index.html
14:23 pdurbin Or do we have to add something to make the graphs?
14:23 donsizemore that was last week; gimme a minute and i'll spin up a VM to poke at
14:23 pdurbin cool, cool
15:02 donsizemore @pdurbin lemme know when you want to look at what prometheus does and doesn't do in its current state and i can send you URL
15:04 pdurbin donsizemore: ready!
15:05 donsizemore starting possibly with https://github.com/prometheus/client_java
15:05 donsizemore now that i look again, it's just a vanilla install here http://irss-dls-buildbox.irss.unc.edu:9090
15:06 donsizemore so i'll point client_java at glassfish and solr for starters
15:10 pdurbin donsizemore: cool but where are the graphs?
15:11 donsizemore there aren't any. i started with the basic framework and got sidetracked. will point client_java at glassfish and solr next?
15:14 pdurbin Hmm, maybe. But maybe it would be nice to have a few graphs enabled from a dashboard or something? This one seems to work, related to memory: http://irss-dls-buildbox.irss.unc.edu:9090/graph?g0.range_input=1h&g0.expr=process_resident_memory_bytes&g0.tab=0
15:14 pdurbin Does that make sense?
15:14 pdurbin I mean, maybe we could start with three graphs or so.
15:15 pdurbin - memory
15:15 pdurbin - disk space
15:15 donsizemore so i'll start with pecking out some built-ins
15:15 pdurbin - cpu
15:15 pdurbin or whatever
15:15 donsizemore the memory seems to be the sticking point ;)
15:15 pdurbin ah :)
15:15 pdurbin well, it's always useful to know when your disk is full :)
15:15 pdurbin standup time
15:19 donsizemore will do
15:49 pdurbin I feel like the Binder folks have nice graphs of their infra. I don't know what they use.
15:49 pdurbin here it is: https://prometheus.mybinder.org/graph
16:28 donsizemore i see nothink
17:06 pdurbin donsizemore: here we go. Please go to https://grafana.mybinder.org and click "Node Activity"
17:17 donsizemore those look really nice
17:21 pdurbin Yeah. But I'm not trying to pile on the work. Do we need Grafana?
17:24 donsizemore mybinder inserts a number of calls in their python to collect data for prometheus https://github.com/search?p=4&q=org%3Ajupyterhub+prometheus&type=Code
17:25 pdurbin Yeah, and I'm seeing some stuff Adam Bien put together to get Glassfish metrics into Prometheus.
17:26 donsizemore https://github.com/AdamBien/firehose
17:27 pdurbin yeah, that's the one I found
17:35 pdurbin Anyway, I fear that if there aren't pretty graphs out of the box people won't see the value in it. I get immediate value from Munin for the operating system level stuff. :)
17:45 donsizemore i'm working on the out-of-the-box part for system-wide numbers but for real information from dataverse you'll want to pepper the codebase for prometheus
17:52 pdurbin Oh yes, absolutely.
17:53 pdurbin Is there a way to create a dashboard of graphs without Grafana?
17:59 donsizemore i mean, i'm just tinkering with prometheus. if you like munin better that's enabled with group_vars as well. but yes it looks like the prettiest graphs would come from grafana
18:02 pdurbin I see.
18:02 pdurbin I don't know what I want. Except that I don't want to waste your time. :)
18:07 donsizemore this is interesting and even fun, but to spin something up at random for basic testing, one may select say go_memstats_heap_inuse_bytes and generate a chart in real time from that VM
18:08 pdurbin Sorry, when you say VM do you mean an instance of the JVM?
18:12 donsizemore let me tinker with jmx_exporter
18:12 pdurbin ok :)
18:13 pdurbin Why do so many metrics start with "go_"?
18:15 donsizemore prometheus is written in go
18:15 pdurbin I was wondering if that's why. :)
18:15 pdurbin So it has nothing to do with go?
18:18 pdurbin I just found this: https://prometheus.io/docs/practices/naming/
19:15 pdurbin donsizemore: anyway, I'm glad you're having fun. :)
19:15 donsizemore i retested the munin branch, BTW - works as advertised
19:16 pdurbin Cool. You mean that the OS level Munin stuff works, right? We'll have to hack on the jstat stuff.
20:55 poiki-mobile joined #dataverse
20:55 poiki-mobile Hey guys, what's up?
21:02 pdurbin poiki-mobile: just got home after picking the kids up from camp. I thought you were on vacation. :)
21:02 poiki-mobile joined #dataverse
21:02 poiki-mobile Yes I am... :-)
21:03 pdurbin tsk, tsk, you should be relaxing :)
21:03 poiki-mobile I am. Did construction work since 8am till 10pm
21:04 poiki-mobile On my way home I listened to the latest airhacks episode...
21:04 poiki-mobile And was dreaming about a Dataverse on steroids ;-)
21:05 pdurbin Hmm, I just finished the second to last.
21:05 pdurbin Which was also good.
21:05 poiki-mobile Yeah
21:06 poiki-mobile Sebastian mentioned Quarkus in that episode
21:06 pdurbin yeah, and fast deployments
21:06 poiki-mobile I was very curious for the current episode all about Quarkus
21:07 poiki-mobile This is so amazing stuff on so many different levels
21:08 pdurbin Yeah, he's covered Quarkus in previous episodes but I'm looking forward to a deep dive.
21:11 pdurbin poiki-mobile: thanks for reaching out to BalusC
21:12 poiki-mobile :-)
21:12 pdurbin I hope you don't mind but I forwarded it to our internal list with my own summary. A summary of my understanding, I mean. I can forward it to you if you like.
21:13 poiki-mobile So many people out there, so I thought I should just ask someone who wrote a book about this stuff... ;-)
21:13 poiki-mobile That's totally fine and that's why I put you and Danny in CC
21:14 pdurbin Cool. I'm just trying to get the word out. It's a good idea.
21:14 poiki-mobile I you don't mind, I would be glad if you could forward it :-)
21:15 pdurbin sure, sent
21:15 poiki-mobile s/I/If/
21:15 poiki-mobile Thx
21:15 pdurbin Please correct any misunderstanding I have. I'm trying to go quickly through a lot of unread email. :)
21:17 poiki-mobile The summary sounds good to me. I wonder about the differences Danny may have...
21:18 pdurbin you mean Gustavo
21:18 poiki-mobile Whoops
21:18 poiki-mobile Yeah
21:19 pdurbin It's hard when you haven't met them. :)
21:19 pdurbin At #dataverse2019 I kept saying that we should have figured out a way to fund your travel. It's a bummer you weren't there.
21:20 poiki-mobile Yeah... better luck next time. ;-)
21:20 pdurbin I already put a bug in my wife's ear about hacktoberfest. :)
21:20 poiki-mobile I hope I did not step on someone's foot in #6070?
21:20 pdurbin Oh and did I tell you I'll probably hopefully be in Portugal in January?
21:20 poiki-mobile Shht, this is not official yet...
21:21 poiki-mobile Oh cool! Where in Portugal? I've been to Feragudo last year.
21:21 pdurbin Lisbon for https://pidapalooza.org
21:23 pdurbin I'm still 4 unread emails away from that issue.
21:23 poiki-mobile Ah nice. I will ask my boss if I should go there instead of going to ETH Z├╝rich...
21:24 pdurbin Would you also go to FOSDEM? It's the weekend after.
21:24 poiki-mobile Oh that's in reach... Brussels is not far away, I can go there by train in about 2 hours
21:25 poiki-mobile maybe 3
21:26 pdurbin I don't think I'm funded to go to FOSDEM though. I'll probably ask. My wife approves it, which is the more important thing. :)
21:26 poiki-mobile Ah, that sounds familiar... :-D
21:27 pdurbin She might even come with me to Portugal. Especially if my parents come to Boston to take care of the kids. I haven't asked yet. :)
21:28 pdurbin It would probably mean less hacking for us though. :)
21:29 poiki-mobile LOL
21:31 pdurbin She's more of a biotech hacker.
21:31 pdurbin She plays God: https://twitter.com/philipdurbin/status/664645552353755136
21:58 pdurbin poiki-mobile: ok, I finally got around to leaving you a comment: https://github.com/IQSS/dataverse/issues/6070#issuecomment-518859597
22:00 poiki-mobile pdurbin I think you misunderstood my explanation ...
22:00 pdurbin whoops
22:00 poiki-mobile Lyncode is relying on v3, Not v2
22:01 poiki-mobile Nothing else than Dataverse Code itself is relying on v2.
22:01 poiki-mobile At least according to mvn dep tree
22:01 poiki-mobile So the WAR will get smaller, as v3 ist already included, but v2 can be left out.
22:02 pdurbin Ok. So you want to remove v2. Would you also delete all the jars in "local_lib/com/lyncode"?
22:02 poiki-mobile Err why should I do so?
22:03 poiki-mobile Isn't this beyond scope for #6070?
22:03 pdurbin I think we're trying to figure out the scope. :)
22:03 poiki-mobile Small chunks... ;-) Dunno where I heard that...
22:03 pdurbin I'm interested in:
22:03 pdurbin - making the war file smaller
22:04 poiki-mobile I totally agree in ditching and dumping that devil local_lib...
22:04 pdurbin - removing local_lib libraries that we have forked
22:04 pdurbin - keeping harvesting (and all features) working :)
22:04 poiki-mobile Nothing that should be kept in a git repo...
22:06 pdurbin Death to local_lib.
22:07 poiki-mobile I think removing local_lib is beyond scope for #6070, but aligned with the goal of making the WAR smaller
22:08 poiki-mobile As this part of local_lib is already relying on v3 and it is "just" Dataverse code that needs changes, it should be pretty straight forward as long as there are no things in use that had breaking changes.
22:08 pdurbin I guess I'm confused. You want to use v3. It's already there. So go ahead and use it. Right?
22:08 poiki-mobile Nonono...
22:08 poiki-mobile Current codebase uses v2.
22:09 poiki-mobile This is a rotten fish.
22:09 pdurbin heh
22:09 pdurbin But I thought you said v3 gets pulled in.
22:09 poiki-mobile v3 ist included as a transient dep of the lyncode
22:09 pdurbin ok
22:10 poiki-mobile So you could use it in the codebase, but you would be breaking the rules of dep managment
22:10 pdurbin heh, what rules? :)
22:10 poiki-mobile actually the Golden rule...
22:10 poiki-mobile Declare want you use.
22:10 poiki-mobile -want +what
22:11 pdurbin So you could declare v3 explicitly. And add a comment saying that we should stop using v2.
22:11 pdurbin Then you won't be breaking the rules. :)
22:11 poiki-mobile Yeah.
22:11 poiki-mobile I could.
22:11 poiki-mobile IMHO this does not make sense.
22:12 poiki-mobile Commons Lang v3 contained some breaking changes.
22:12 pdurbin What would you rather do?
22:12 poiki-mobile I don't think Dataverse is affected by those.
22:13 poiki-mobile So this merily bouls down to replacing some imports and changing the version in POM
22:13 poiki-mobile boils
22:13 pdurbin But doesn't harvesting require a patched jar? Would your change break harvesting?
22:14 poiki-mobile Again: lyncode does not use v2, but relies on v3.
22:14 poiki-mobile It is just our code using the v2 lib as far as I can tell from mvn dep tree plugin
22:15 poiki-mobile I would just swap out v2 and use v3 everywhere.
22:15 pdurbin Ok, so in your pull request you would not remove our patched version of lyncode. How much Dataverse code uses v2? How much testing would QA have to do?
22:16 poiki-mobile Let me switch to my laptop to get some numbers...
22:16 pdurbin ok
22:17 poikilotherm joined #dataverse
22:17 pdurbin and sorry, I'm a little distracted trying to catch up on email
22:18 pdurbin 18 unread github issue threads
22:18 pdurbin poikilotherm: what's the benefit or removing v2? A smaller war file? How much smaller?
22:19 poikilotherm It's always benefical ripping out a library not updated since 2011
22:20 poikilotherm Let me check on the size
22:20 pdurbin If we're going to work on ripping out old things, let's put our energy into ripping out old app servers. :)
22:21 poikilotherm This is a low hanging fruit...
22:21 poikilotherm commons.lang is used in 87 class files
22:22 poikilotherm find src -name '*.java' -exec grep -Hi "commons.lang" {} \; | cut -f1 -d":" | sort | uniq -c | wc -l
22:22 poikilotherm Oh and lang3 is already used in 3 files
22:22 poikilotherm Err... 5
22:22 pdurbin Sound like a lot. Would you write up a "how to test" doc for QA?
22:23 poikilotherm There should be no exhaustive testing necessary. Breaking changes will lead to compilation failures AFAIK
22:23 poikilotherm But of course I can provide a list of features affected
22:24 pdurbin That would be helpful. But should I go look at the list I made for you? :)
22:24 poikilotherm Hmm maybe I can even shorten that list to things not covered by tests
22:24 poikilotherm Your board?
22:25 poikilotherm (column)
22:25 pdurbin Yeah, I made a column for you. You told me I could. :)
22:25 pdurbin I forget what's on there.
22:25 pdurbin Something about spinning up arbitrary brankes on k8s.
22:25 pdurbin branches*
22:25 poikilotherm Yeah
22:25 poikilotherm I am currently testing this locally
22:25 poikilotherm Redeployment times ARE DRIVING ME NUTS
22:25 pdurbin Yeah, that's the only thing in your column: https://github.com/orgs/IQSS/projects/3#column-5999026
22:26 poikilotherm I have to wait ~8 min
22:26 poikilotherm for
22:26 poikilotherm every
22:26 poikilotherm fricking
22:26 poikilotherm single
22:26 poikilotherm small
22:26 poikilotherm change
22:26 pdurbin heh
22:26 poikilotherm Build is fast
22:26 poikilotherm But fear deployment times
22:27 pdurbin Should we take something tiny like https://github.com/pdurbin/javaee-docker and put all the goodies and good practices into it?
22:27 poikilotherm That's why I am so exited about Quarkus BTW ;-)
22:27 pdurbin ...goodies including quarkus if you want
22:28 poikilotherm Adam mentioned some showcase project for Quarkus in the last episode
22:28 pdurbin ah, ok
22:28 poikilotherm http://www.adam-bien.com/roller/abien/entry/quarkee_the_java_ee_stic
22:28 pdurbin mvn archetype:generate -DarchetypeGroupId=com.airhacks -DarchetypeArtifactId=quarkee-archetype
22:28 pdurbin ok
22:29 pdurbin Sure, we could use that as a starting point.
22:30 pdurbin We could call it dataverse-next or something. :)
22:30 poikilotherm LOL
22:31 poikilotherm https://developers.redhat.com/blog/2019/04/12/migrating-java-applications-to-quarkus-lessons-learned/
22:32 poikilotherm Maybe one should think about getting of Glassfish/Payara and move to Wildfly...
22:32 pdurbin A long time ago I created https://github.com/IQSS/iqss-javaee-template
22:32 poikilotherm Getting there is most likely making it a lot easier to have sth. like Quarkus get used, as it uses Wildfly under the hood
22:33 poikilotherm WOW! That's cool :-)
22:34 pdurbin :)
22:34 pdurbin It's funny to see icefaces in there.
22:34 poikilotherm Ancient archetypes from the past are amazing :-)
22:35 poikilotherm Man this is from 2012 and look where we are today
22:35 pdurbin At the time there were two projects at IQSS that used very similar stacks. Dataverse is the open source one.
22:36 poikilotherm I just browsed through the logs for today...
22:37 poikilotherm Saw you and donsizemore talking about Prometheus...
22:37 poikilotherm Just recently I began to play with Prometheus and Grafana for my laptop ;-)
22:37 poikilotherm Planning to add a small howto to dataverse-kubernetes later on, as this is state of the art
22:38 poikilotherm Especially for K8s deployments
22:38 poikilotherm There are more options obviously ;-)
22:39 poikilotherm I already have some experience with the other classic collectd+Influx+Grafana
22:39 pdurbin oh, great!
22:39 pdurbin speaking of ancient, I still use Munin on my home server (and I love it)
22:40 poikilotherm We could easily create a dashboard in Grafana and share it on the Grafana website
22:40 poikilotherm And we could easily share a prometheus config
22:41 poikilotherm You could already have a look at the data with prometheus itself, but Grafana has a lot of more options for graphing
22:41 poikilotherm And there is a good chance these days there is already an installation of Grafana present
22:43 pdurbin Did you see Don's pull request to add Prometheus to dataverse-ansible?
22:45 poikilotherm Nope. I don't follow dataverse-ansible very closely...
22:45 pdurbin poikilotherm: please feel free to leave a review: https://github.com/IQSS/dataverse-ansible/pull/96
22:45 poikilotherm pdurbin should we write an update to #6070?
22:46 pdurbin Maybe, I'm sorry I got it wrong.
22:46 pdurbin I'd like to find a better use of your time, honestly. Monitoring would be great.
22:47 pdurbin A reproducible test case for a question for BalusC on Stack Overflow would be great.
22:47 poikilotherm I need to work on #6000 first... ;-)
22:47 pdurbin I have some "help wanted: python" stuff on my board if you'd like to take a look at any of that.
22:47 poikilotherm I have sth. working, but wanted to have tests, too
22:48 pdurbin Some help with Jenkins would be great.
22:48 poikilotherm Hihihi
22:49 poikilotherm Go get some funding, I try to get a month or two of and work for IQSS on whatever you want :-D
22:49 pdurbin :)
22:49 poikilotherm (Although I doubt my boss would do that, at least not until we are in production)
22:50 pdurbin Let's have you work on something your boss cares about and that I care about. :)
22:50 pdurbin I'm sure there's plenty of common ground. :)
22:50 poikilotherm I hope so :-D
22:51 poikilotherm Can you leave a comment on #6070? I really need to get some sleep... 1am over here...
22:52 pdurbin I'll leave myself a reminder to do so.
22:52 poikilotherm Thank you so much! Awesome :-)
22:52 pdurbin reminder made :)
22:53 poikilotherm Have a good night - maybe read you tomorrow... ;-)
22:53 pdurbin get some sleep!
22:53 pdurbin :)

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.