Time
S
Nick
Message
01:16 jri joined #dataverse
04:16 jri joined #dataverse
07:16 jri joined #dataverse
07:37 MrK joined #dataverse
07:50 bamouni joined #dataverse
08:11 jri joined #dataverse
08:27 juancorr joined #dataverse
10:13 pdurbin
bamouni: welcome!
10:19 pdurbin
How is everyone's day going?
10:37 pdurbin
Dataverse 4.12 is out: https://github.com/IQSS/dataverse/releases/tag/4.12
11:00 bamouni56 joined #dataverse
11:06 poikilotherm joined #dataverse
11:07 poikilotherm
Mornin' pdurbin. About my hassle with JVM options... I needed a quick workaround for our cluster to configure S3 options where I myself introduced lots of options with dashes... :-/ My bad... So I came up with this https://github.com/IQSS/dataverse-kubernetes/issues/11#issuecomment-478909197 for the time being.
11:14 pdurbin
basically tr '_' '.' it looks like. seems fine
11:16 bamouni56 left #dataverse
11:16 poikilotherm
YEah, but sed s#__#-#g
11:17 pdurbin
poikilotherm and MrK: Are you familiar with scripts like scripts/database/create/create_v4.12.sql ? I'm asking because we're thinking about no longer creating them now that we have Flyway.
11:18 poikilotherm
Err... the content of that file is in the flyway migration scripts, too?
11:19 pdurbin
I called these "create" scripts "raw material" for docker/kubernetes over at https://github.com/nds-org/ndslabs-dataverse/issues/8#issuecomment-442637425
11:20 poikilotherm
Ah I see - this is a creation script when you start from scratch, right?
11:21 MrK
pdurbin: In our fork we are just using flyway anyway :p
11:21 pdurbin
Right, I've been thinking that you could spin up a Docker image of just Postgres without Glassfish. Just load up the "create" script into that Postgres container. But if no one wants this, we could stop making these "create" scripts.
11:21 poikilotherm
Hmm has there been a use case for that?
11:22 pdurbin
Well, when I was working with RedHat on getting Dataverse running on OpenShift they would have used create scripts like this. But we didn't have them then, unfortunately.
11:23 poikilotherm
In my k8s stuff I start with a plain and clean postgres container, which will contain the database structure as soon as glassfish is starting
11:24 pdurbin
Right. The Red Hat guy was a little grumpy about needing Glassfish to populate teh postgres schema.
11:24 pdurbin
It slowed everything down.
11:24 poikilotherm
Yeah, but they are buying it by having another dependency to take care of...
11:25 MrK
Well if you would want a modern way then container should be embedded, there would be no need for external glassfish
11:25 poikilotherm
And migrations are not possible with this, too
11:26 pdurbin
Why wouldn't migrations be possible?
11:26 poikilotherm
That create scripts seems only applicable when you start with an empty database
11:26 pdurbin
Oh! Really?
11:27 poikilotherm
Lots of CREATE in there
11:27 poikilotherm
And I compared v4.11 and v4.12 scripts - its more or less the same script
11:27 poikilotherm
Except for the changes from 4.11 to 4.12
11:29 poikilotherm
But again - I just gave it a quick look
11:30 pdurbin
Interesting. This script was removed from 4.12. It operated on the "create" scripts: https://github.com/IQSS/dataverse/blob/v4.11/scripts/database/dbupgrade.sh
11:31 pdurbin
Oh, wait, I'm wrong. The 4.12 tag is missing a v.
11:31 poikilotherm
Oh oh
11:32 poikilotherm
Sounds like I need to check that... Would break my Dockerfile :-(
11:33 poikilotherm
Man, good catch
11:33 poikilotherm
This will break my Dockerfile
11:33 poikilotherm
Meeeh
11:33 poikilotherm
No simple number swapping but changing the URL , too
11:34 poikilotherm
(I don't like having a script dangling around with the version string...)
11:34 poikilotherm
Chances you guys might change the upstream release?
11:35 poikilotherm
You can see the URL stuff here: https://github.com/IQSS/dataverse-kubernetes/blob/df25c3cd0f4bc83ae83b3bca039c0eeb30d845c5/dataverse-k8s/Dockerfile#L27
11:35 pdurbin
I just created an issue: https://github.com/IQSS/dataverse/issues/5709
11:35 pdurbin
Please feel free to leave comments.
11:36 pdurbin
Anyway, I was wrong. The dbupgrade.sh script is still there.
11:37 poikilotherm
Thx for the issue, just left a comment :-)
11:37 pdurbin
MrK: I hear you about embedded glassfish and your fork but I'm still trying to think through whether we should bother making these "create" scripts anymore.
11:38 poikilotherm
Maybe create an issue asking for a use case?
11:38 pdurbin
Maybe.
11:38 poikilotherm
(And mention people you know that might be affected)
11:38 pdurbin
It's the "2. Save the EJB Database Create Script" step in our release process: http://guides.dataverse.org/en/4.12/developers/making-releases.html#save-the-ejb-database-create-script
11:38 pdurbin
We could simply take out that step.
11:39 poikilotherm
Ah that one again. I hope I have some time in the near future to get rid of the database timers for 4.13
11:43 pdurbin
Well, that's the "create" script I'm talking about. Yes, it has the word "EJB" in the name but I don't think it has much to do with the EJB timers.
11:43 poikilotherm
It does a lot more things than just creating EJB timer tables :-D
11:44 pdurbin
Anyway, the use case described for the create script is our update script. I thought the create script would be useful for someone who wants to spin up postgres in a separate docker container. But it seems like no one is doing that.
11:52 pdurbin
Red Hat probably would have tried using them but they aren't actively involved at the moment.
11:57 pdurbin
I'll probably check in with Craig. That's who I was talking to about raw material in that issue I linked earlier.
11:58 pdurbin
poikilotherm: that said, I expect Craig would be happy to switch from his Docker images to yours or Slava's if they work in his "workbench". NDS Labs Workbench: https://github.com/IQSS/dataverse/issues/4152
11:58 poikilotherm
Yeah, sound reasonable :-)
11:58 poikilotherm
Sure. Any feedback is appreciated :-D
12:00 poikilotherm
pdurbin I am AFK for a while. See you later.
12:01 pdurbin
o/
12:01 pdurbin
MrK: by embedded glassfish, you mean a fat war with glassfish inside, right?
12:02 MrK
pdurbin: more like jar but yeah :P
12:02 pdurbin
sorry, far jar
12:02 pdurbin
bah
12:02 pdurbin
fat jar
12:02 MrK
jar jar
12:03 pdurbin
heh
12:03 pdurbin
The Dataverse war file is already 188 MB . How big is too big? I imagine adding Glassfish would make it quite a bit bigger.
13:09 donsizemore joined #dataverse
13:10 donsizemore
@pdurbin anything greater than 640KB
13:10 * donsizemore
ducks
13:12 pdurbin
!
13:12 pdurbin
Bigger is better. More features! ;)
13:12 pdurbin
It's like owning a mansion.
13:13 pdurbin
And paying for someone to clean it.
13:18 donsizemore
@pdurbin don't look now, but another dataverse is down
13:18 pdurbin
!
13:18 pdurbin
is it mine?
13:19 donsizemore
i'm sticking my little print() statement back in download.py to accuse the culprit
13:19 pdurbin
sounds python 3 compatible. I approve
13:20 donsizemore
my coworkers never liked getting system barf overnight; i like to see nightly output to know things ran properly
13:21 pdurbin
Sure. Please feel free to propose code changes to make it easier to run that thing in production. You could contribute your cron job if you want.
13:22 donsizemore
somebody kicked their glassfish instance; we were getting 503s a minute ago
13:26 pdurbin
Gotcha. Hey, do you feel like making a pull request to update a future version of http://guides.dataverse.org/en/4.12/installation/config.html#metricsurl to no longer mention miniverse? I was thinking we could link to the reporting tools page in the Admin Guide instead.
13:28 donsizemore
yes though i feel dataverse-metrics, as 'IQSS Official' should be at the top of that page(?)
13:28 pdurbin
I humbly added it to the bottom but sure, I don't mind it being at the top. We could also clean up those URLs a bit.
13:29 donsizemore
maybe make a distinction between IQSS vs. community-maintained, or just put them in alpha order
13:31 pdurbin
I don't have a strong opinion about it. In general, I like promoting work from the community.
13:32 pdurbin
donsizemore: meanwhile, I just opened this: https://github.com/IQSS/dataverse-ansible/pull/54
13:33 donsizemore
@pdurbin thanks for doing that, but i'm not done yet =)
13:33 pdurbin
no worries, please feel free to close it
13:33 donsizemore
@pdurbin also will add previewer and letsencrypt but jon is out this week
13:34 donsizemore
@pdurbin hey, i never met a pull request i didn't like
13:34 pdurbin
so you're going to all his meetings?
13:41 donsizemore
i'm realizing how many people come to him for stuff
13:48 donsizemore
on 4.9.4 the QDR previewer curl commands return Unknown reserved word: {datasetId}
14:00 pdurbin
donsizemore: yeah, those previewers will only work on Dataverse 4.11 or higher.
14:01 pdurbin
donsizemore: I just sent you an email about this
14:03 pdurbin
How's it going with adding files to sample data? If it's difficult we can defer this. I'd like Mike to take a look at what you've got. It's working for datasets and dataverses.
14:03 donsizemore
@pdurbin i may submit a pull request to add that minimum =) and files were where i left off yesterday. next on my list
14:04 pdurbin
sounds good. thanks!
14:11 donsizemore
speaking of... known files to known datasets? random dataset ID for each?
14:11 donsizemore
i stole your files from scripts/search/data/tabular
14:12 pdurbin
I would associate certain files with certain datasets. If that makes sense. Not random.
14:13 pdurbin
My "Spruce Goose" dataset gets an image with some trees, for example. And it's in the "Trees" dataverse.
14:14 pdurbin
I decided that birds and trees are less controversial that Simpsons characters or whatever.
14:14 pdurbin
But I'll be happy when someone steps in and makes some real looking datasets.
14:15 pdurbin
Thanks for making this customizable.
14:17 donsizemore
speaking of simpsons characters: https://www.thepoke.co.uk/2019/04/02/the-obituary-of-matt-groenings-mum-explains-a-lot-about-the-simpsons/
14:18 pdurbin
heh, Wiggum
14:49 donsizemore joined #dataverse
15:22 donsizemore
@pdurbin "real-looking datasets" means full examples with json metadata, yes?
15:35 pdurbin
yeah, but I don't expect you or I to come up with those examples
15:41 donsizemore
@pdurbin well, i just made some. running in vagrant to check for typos
15:56 pdurbin
nice
17:02 donsizemore
@pdurbin knock knock?
17:11 pdurbin
donsizemore: who's there
17:31 jri joined #dataverse
17:34 andrewSC joined #dataverse
18:47 donsizemore
@pdurbin eh, i was going to ask some silly question but i forget what it was now
18:48 pdurbin
:)
19:03 donsizemore
@pdurbin also test the 37 branch if you have time
21:11 donsizemore joined #dataverse
21:38 pdurbin_m joined #dataverse
21:38 pdurbin_m
donsizemore: sure, please go ahead and merge https://github.com/IQSS/dataverse-ansible/pull/57
21:43 pdurbin_m
lol. "has it really come to this"
21:59 pdurbin_m
if I could merge from my phone I would :)
