IQSS logo

IRC log for #dataverse, 2020-12-09

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
04:01 sivoais joined #dataverse
05:21 JonathanNeal_ joined #dataverse
07:41 Virgile joined #dataverse
07:50 juancorr joined #dataverse
12:24 donsizemore joined #dataverse
13:31 stefankasberger joined #dataverse
13:39 donsizemore joined #dataverse
14:15 pkiraly joined #dataverse
14:39 pameyer joined #dataverse
14:41 pameyer @donsizemore - did the cent8 news make it to your radar?
14:56 nightowl313 joined #dataverse
15:32 donsizemore @pameyer 8.3? already done
15:34 pameyer nope - the EOL date change and switch to centos stream
15:35 Virgile hi there, about centos, did you see this ?
15:35 Virgile https://lists.centos.org/pipermail/centos-announce/2020-December/048208.html
15:35 pameyer cent8 now has an earlier EOL than cent7
15:36 pameyer @Virgile - thanks for the link, yes
15:37 pameyer seemed like it might be info relevant to what OS to recommend for dataverse installations
15:37 stefankasberger Announcement: pyDataverse is now also part of the GDCC family. :) https://github.com/gdcc/pyDataverse
15:37 stefankasberger And a release will follow the next days.
15:38 Virgile \o/
15:39 pameyer congrats stefankasberger
15:40 donsizemore @pameyer oh, yeah, the streams thing. with no munnies whut i'm gonna do
15:43 pameyer I'm kinda leaning towards sticking with cent7 and waiting to see how things develop.  maybe my paranoia, but recommending a rolling release to end users seems like it might be asking for trouble
15:50 donsizemore for my part, everything we call "production" is RHEL. everything on CentOS is... free.
15:59 pdurbin joined #dataverse
15:59 pdurbin Whoa. Full house. Hey everyone. ­čĹő
16:00 pdurbin Huh. "cent8 now has an earlier EOL than cent7"
16:00 pameyer pdurbin: yup
16:00 pdurbin stefankasberger: excellent news!
16:01 pameyer seemed to be worth discussion (but stefankasberger's news is cooler :) )
16:01 pdurbin No, no, it's all worth discussing. Just one thing at a time, ideally. Sorry to cross the streams. :)
16:03 pameyer :)
16:05 pameyer the cent8 / centos streams thing is new enough that I'm still not sure what I think about it.  but I had the sense that not all dataverse installations had rhel
16:05 pdurbin poikilotherm: it looks like https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-7463/11/ failed because the ec2 instance couldn't be reached. So I just clicked "build now" to see what happens.
16:06 pameyer so maybe more a heads up than discussion - shutting up now ;)
16:09 pdurbin news to me!
16:16 donsizemore @pameyer I, BTW, blame IBM.
16:22 donsizemore it would have been nice for say 9 rather than to reduce 8's EOL from 2029 to 2021
16:24 pameyer yeah - probably related to IBM's split / cloud reorg thing
16:25 pdurbin Should we revert the Dataverse guides from 8 to 7? And all the code. What a pain.
16:26 pameyer maybe - by Murphy's law, the announcment happened after all the guides/code stuff had been changed
16:26 pdurbin yeah
16:26 pdurbin When is the next LTS? Or will it be like this forever?
16:26 donsizemore I'm converting dataverse5.odum.unc.edu to CentOS 8 Stream now
16:27 pameyer I don't think there is a next LTS
16:27 donsizemore one simply installs centos-release-stream, then calls dnf distro-sync
16:27 donsizemore there are currently conflicts with openblas-devel, but dnf distro-sync --nobest cleared that up
16:28 pdurbin If there isn't a next LTS, there's no sense in holding on to cent7, right?
16:28 donsizemore @pdurbin there will be a RHEL 8 and a RHEL 9
16:28 donsizemore @pdurbin it's just CentOS which is moving out from behind RHEL (repackaging) to become an intermediary testing ground in between Fedora and RHEL.
16:29 donsizemore I'm comfortable with that. It was always free software with no support.
16:29 pdurbin This explains the flurry of activity of our devops group moving systems from centos to rhel.
16:29 donsizemore this was news to me.
16:30 nightowl313 yikes, our installations are centos 8 ... will we have an issue?
16:30 pameyer @nightowl313 probably not until end of 2021, but probably something to plan for soon-ish
16:31 donsizemore IBM wants you to buy RHEL licenses
16:31 pameyer @donsizemore - makes sense about RHEL 8,9 ; I'd been focused on the centos changes
16:31 donsizemore @pameyer follow the munny
16:32 donsizemore @nightowl313 I'm converting dataverse5.odum.unc.edu from CentOS 8 to CentOS stream. about to reboot it.
16:33 nightowl313 okay ... thanks for the heads up!
16:34 nightowl313 @donsizemore let me know how that goes ... wonder if should do the same
16:34 pameyer @donsizemore yup
16:36 donsizemore It rebooted cleanly. No kernel change but updated kmod-kvdo and glib packages so I rebooted
16:37 donsizemore nothing terrible yet: https://dataverse5.odum.unc.edu/
16:38 donsizemore SElinux is ticked off at collectd but that's just something to fix
16:38 donsizemore I did have to give dnf the --nobest flag during distro-sync but the R plenum probably didn't get much testing
16:39 donsizemore and I imagine that will be fixed in time
16:39 pameyer sounds promising
16:42 nightowl313 I better make a plan ... will try on our test site
17:28 nightowl313 curious if it was ever considered to run dv on ubuntu?
17:53 poikilotherm left #dataverse
17:53 pdurbin nightowl313: some installations run on Ubuntu. I can't remember which ones.
17:54 poikilotherm6 joined #dataverse
17:56 donsizemore @pameyer @nightowl313 back from a quick lunch and puppy-dog walk.
17:56 donsizemore tl;dr: if you have any trouble with CentOS streams at present it will be in R, which is always a crap-shoot during upgrades
17:56 donsizemore if you're thinking Ubuntu I'd think about Debian 10 instead
17:57 Virgile joined #dataverse
17:58 donsizemore I personally have no qualms about CentOS moving in between Fedora and RHEL, but I work for a public university in which everything is important despite there never being enough money.
17:59 nightowl313 i think we definitely want to keep with what is more generally used/recommended/supported ... prob will stick with centos stream or rhel 8
18:00 donsizemore @nightowl313 Dataverse should run just fine on Ubuntu; the big deal will be how long a given distro supports JDK 1.8.0 (which Dataverse requires, though I've tested Dataverse with Java 11)
18:01 nightowl313 just curious about ubuntu .. all of our other VMs and instances are ubuntu
18:01 nightowl313 maybe I'll try it in test one day =)
18:11 pameyer wouldn't expect any problems with ubuntu - _maybe_ locations of systemd unit files and dependency package names
18:14 pameyer ubuntu LTS releases (for non-dataverse things) haven't seemed to cause any server issues related to their ubuntu-ness, other than making sure ansible roles have the right package names for the distro
18:14 poikilotherm6 Usually the JVM part is not a problem on any distro. But the native stuff like jHove etc _might_ run into troubles.
18:15 poikilotherm6 And isn't there a dep on ImageMagick? I remember troubles with that package on some distros. (Just a bell in the back of my head)
18:17 poikilotherm6 Ah it was ImageMagick having troubles to convert PDF on some platforms due to a strange policy installation
18:20 pameyer I'd forgotten ImageMagick - pretty sure all the integration tests pass without it through
18:28 poikilotherm6 For all of us dealing with large piles of JSON on CLI: go checkout https://github.com/antonmedv/fx Saw it on Twitter today...
18:36 pameyer what brought that one to your attention?
18:37 poikilotherm6 climagic retweeted it an I follow them
18:38 pameyer joined #dataverse
18:38 poikilotherm6 And some weeks ago I was dealing with jq filters and other stuff to make at least some sense from the output I was facing. Collapsible json on CLI would have helped with that
18:39 poikilotherm6 (Of course, one could open with an editor and use folding feature of, say, atom sublime you name it, but this looks easier and quicker to use)
18:41 pameyer vim will do collabsible json on cli ;)
18:41 pameyer good find
18:41 pameyer jq is great for simple things, but sometimes it feels like complex stuff takes me longer than it should (likely a problem on my end, but always good to find tools to help)
20:40 kaitlin joined #dataverse
20:42 kaitlin Hey all, has anyone run into an issue with originalfilesize missing for files in the db? We are seeing it in some older tabular files and it's causing internal server errors on the dataset page. This started with our recent upgrade to 5.1 from 4.19.
20:45 kaitlin It seems that the UI change in https://github.com/IQSS/dataverse/issues/6118 may be related for us.
20:48 pameyer you're also seeing null vs zero?
20:48 kaitlin They are null in the database
20:48 kaitlin the dataset page itself doesn't load
20:51 pameyer if this is "production is broken", a db change from null -> 0 might get things operational (after a query to know what needs fixing later)
20:51 pameyer if not, I'd recommend waiting until pdurbin (or somebody other than me) has better ideas - I'm not fully up to speed on tabular data stuff
20:53 kaitlin thank you, yes, it is a "production is broken" case for several datasets! We noticed uningesting and reingesting was one way around it, but that would be a lot of 2800+ files.
20:53 kaitlin a lot for*
20:54 pameyer that would be a lot of work
20:55 kaitlin Our sysadmin is also thinking that updating originalfilesize with filesize value from datafile table for same file id might be another option
20:55 pameyer I'm not 100% sure that the sql update would work, but it would be an easy thing to try (and if you've recently upgraded, your database backups are probably good)
20:55 pameyer agreed - if you have better data than 0, that would be closer to what users are expecting
21:15 kaitlin thanks pameyer!
21:25 pameyer you're welcome kaitlin - good luck!
21:28 kaitlin a quick follow up - I found this API that might help us here so this might be the way to go! https://guides.dataverse.org/en/5.1/api/native-api.html#datafile-integrity
21:34 kaitlin joined #dataverse
21:34 kaitlin So I think that anyone that didn't run that API call in the 4.10 upgrade might run into the same problem we did.
22:06 pdurbin sorry, was on a lot of calls this afternoon
22:07 pameyer zoom fatigue is real
22:08 pameyer and I'm glad your first message wasn't "no, that's a horrible idea" ;)
22:08 pameyer ^ since kaitlin already disconnected and may have tried it
22:10 pdurbin There was some related chatter at https://github.com/IQSS/dataverse/issues/6118#issuecomment-742023678
22:10 pdurbin I think they're all set. Hope so.
22:11 pdurbin Anyway, taking off. See you tomorrow.
22:11 pdurbin left #dataverse
22:12 pameyer have a good night

| Channels | #dataverse index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.