Time
S
Nick
Message
02:59 jri joined #dataverse
05:59 jri joined #dataverse
07:14 jri joined #dataverse
08:07 cdsp-rmo joined #dataverse
10:14 poikilotherm joined #dataverse
10:26 pdurbin joined #dataverse
11:19 juancorr joined #dataverse
11:48 donsizemore joined #dataverse
12:34 pdurbin
I hope everyone had a good weekend.
12:35 * poikilotherm
reading up logs
12:35 poikilotherm
Yes, I had ;-)
12:37 poikilotherm
I just tested PhraseApp.com out of curiosity...
12:37 poikilotherm
Mixed results :-/
12:37 pdurbin
Ah. Thanks for looking into that.
12:37 pdurbin
juancorr: have you heard of these apps?
12:38 pdurbin
cdsp-rmo: are you involved in translation at all?
13:09 poikilotherm
pdurbin: as promised, I opened an issue about the microprofile stuff https://github.com/IQSS/dataverse/issues/5187
13:10 juancorr
pdurbin: Sorry I do not know these apps. I only use the diff scripts.
13:11 pdurbin
poikilotherm: this is great! Thanks!
13:12 pdurbin
juancorr: no worries and thanks for providing that script!
13:13 pdurbin
poikilotherm: how do you feel about working on https://github.com/IQSS/dataverse/issues/4172 as what seems to me to be the next logical step?
13:14 poikilotherm
I think those issues are closely related.
13:14 poikilotherm
Well, you can use Payara Micro 4.1.2, but we really should get things running on modern Java EE 8 based servers
13:15 pdurbin
Yeah. I listen to http://airhacks.fm and Adam Bien has mentioned a few times that Payara "full" and Payara "micro" are pretty much the same, performance-wise, so I think we should start with "full" and see if we can eventually get Dataverse to squeeze into skinny jeans.
13:17 poikilotherm
I already went through `grep -r "import javax" src | cut -f2 -d: | cut -f1,2 -d. | cut -f2 -d" " | sort | uniq -c`...
13:17 poikilotherm
1 javax.activation
13:17 poikilotherm
22 javax.annotation
13:17 poikilotherm
47 javax.batch
13:17 poikilotherm
9 javax.crypto
13:17 poikilotherm
338 javax.ejb
13:17 poikilotherm
9 javax.enterprise
13:17 poikilotherm
212 javax.faces
13:17 poikilotherm
7 javax.imageio
13:17 poikilotherm
193 javax.inject
13:18 poikilotherm
11 javax.jms
13:18 poikilotherm
359 javax.json
13:18 poikilotherm
20 javax.mail
13:18 poikilotherm
9 javax.naming
13:18 poikilotherm
1048 javax.persistence
13:18 poikilotherm
107 javax.servlet
13:18 poikilotherm
79 javax.validation
13:18 poikilotherm
308 javax.ws
13:18 poikilotherm
93 javax.xml
13:18 poikilotherm
Most of this seems to be included in the Payara Micro server
13:18 poikilotherm
(Which is extending the Microprofile)
13:18 poikilotherm
This is still less than the full profile
13:19 pdurbin
Right, but let's get Dataverse running on Payara "full" first. We use stuff like JMS, so we need "full".
13:23 poikilotherm
Just added a comment to the issue about that... :-D
13:24 pdurbin
looks great
13:25 poikilotherm
Payara Micro offers JCA support, which can be used for JMS
13:27 poikilotherm
I had no chance yet to find out when using a "Full profile" if there is a broker included or if this is external anyway.
13:28 poikilotherm
Ah I forgot... The broker might be offered by Glassfish directly
13:28 poikilotherm
(embedded)
13:28 pdurbin
I'm not sure what JCA is but smaller steps are better. Payara Full, in my opinion.
13:28 pdurbin
Does Payara Micro offer any feature that Payara Full doesn't have?
13:29 poikilotherm
Nope - it's just smaller and you can embed it.
13:29 poikilotherm
I not sure if you can build a runnable JAR with the "full" server
13:29 pdurbin
I'm pretty sure you can embed Full also.
13:29 poikilotherm
+am
13:33 pdurbin
I don't know. It's hard to tell the Payara website. Do we need it to be embedded right now? Can that be a future thing?
13:37 poikilotherm
Sure. I just thought I'd keep an eye on both
13:38 pdurbin
Yep. Good idea. You see what I mean about the smallest steps possible, right?
13:47 poikilotherm
Sure ... :-D
13:52 pdurbin
good :)
13:52 pdurbin
poikilotherm: so do you want to work on the Payara issue? :)
13:53 poikilotherm
Yes, please assign me ;-)
13:57 pdurbin
poikilotherm: done! Thanks! Heads up to donsizemore who opened that issue.
14:00 poikilotherm
Ah guys... I just walked through the 500 lines of depenendcies... That really plus the never ending tree of classes could use some maven based modulization of the codebase, right?
14:03 pdurbin
I'm sure you're right but what file(s) are you looking at? I'm confused.
14:03 poikilotherm
pom.xml
14:04 poikilotherm
That's quite a huge block of stuff, sometimes unknown where this is used.
14:04 pdurbin
yeah
14:07 poikilotherm
Maybe https://maven.apache.org/guides/mini/guide-multiple-modules.html could help`
14:07 poikilotherm
Could make the codebase tiddier, too...
14:08 poikilotherm
Dunno... Don't know the codebase so well
14:08 poikilotherm
pdurbin: why did you close https://github.com/IQSS/dataverse/pull/4244
14:09 poikilotherm
Where there issues holding you guys back or just other stuff getting higher prio?
14:10 donsizemore
@pdurbin could you move 5173 back out of code review? just until i rename those variables in both 5173 and dataverse-ansible and test again
14:10 poikilotherm
https://github.com/IQSS/dataverse/issues/4248#issuecomment-403778564 is not very enlightning...
14:11 poikilotherm
Did I get you right, that this only stopped because you rejected the usage of glassfish?
14:11 poikilotherm
(Which is abolutely right as Oracle dropped support after all)
14:14 pdurbin
poikilotherm: at https://github.com/IQSS/dataverse/pull/4244#issuecomment-342799756 it looks like I said the issue got moved out of the sprint. So I closed the pull request. I don't like having ancient pull requests lying around.
14:14 pdurbin
donsizemore: sure, back to community dev it goes
14:15 poikilotherm
Right, I read that. But why was it removed from the sprint?
14:16 pdurbin
I'm thinking.
14:16 poikilotherm
:-D
14:17 poikilotherm
I see the smoke frome here... ;-)
14:17 pdurbin
I don't think we wrote down why.
14:17 pdurbin
Often we get busy with other stuff.
14:17 poikilotherm
Alright, that's fine :-)
14:18 poikilotherm
I'll just see if I can reapply your patches to the current codebase and try to get things rolling
14:25 pdurbin
poikilotherm: ah, that pull request I closed was for going from Glassfish 4 to Glassfish 5. Are you working on that or working on getting Dataverse to run on Payara?
14:27 poikilotherm
Well as Payara 5 ~ Glassfish 5, I thought I would first look up what happened then...
14:28 poikilotherm
But as Glassfish 5 is unsupported by Oracle, I will definitly go for Payara 5
14:28 poikilotherm
(As Payara took of where Oracle phased out)
14:28 pameyer joined #dataverse
14:31 poikilotherm
pdurbin: are there many people asking for the vagrant stuff? IMHO this seems pretty outrun by Docker
14:32 poikilotherm
(which seems faster, too - building vagrant right now and it takes AGES)
14:33 poikilotherm
I would rather come up with a proper docker compose file like for the DCM
14:33 pameyer
poikilotherm: I've been known to refer to docker-aio as completely opposite of containerization best practices
14:34 pameyer
always the issue of where to draw the abstraction boundaries though
14:34 poikilotherm
pameyer: I didn't want to speak it out loud, but as you mention it...........
14:35 pameyer
:) it's pretty much docker done exactly wrong. but it solves the problem it was intented to solve
14:35 poikilotherm
Well the vagrant up took 5 minutes and still downloading...
14:35 poikilotherm
Even docker-aio is faster :-D
14:36 pameyer
my assumption is that long term it would make sense to have docker-aio replaced by something more in line with containerization best practices. but doing it that way would have slowed down implementation time
14:37 pdurbin
poikilotherm: I like having both Vagrant and Docker. I guess it's just a comfort thing for this old dinosaur.
14:38 poikilotherm
:-D
14:38 poikilotherm
It kind of doubles the maintenance...
14:38 pdurbin
You're right.
14:38 poikilotherm
Maybe you could run a docker-compose based setup in a vagrant box?
14:38 pameyer
pdurbin: both docker and vagrant can be fiddly at times - but if all goes well, they're fiddly at *different* times
14:39 poikilotherm
This ideally takes the advantages from both worlds
14:39 pdurbin
Let me sleep on it. I'm not sure I'm ready to walk away from Vagrant. It's been a long term relationship.
14:39 poikilotherm
Sure. Talk your time, granpa... :-P
14:40 pdurbin
Thanks. I've been using Vagrant since before Docker existed.
14:40 poikilotherm
(Just referencing the dinosaur from above... ;-) )
14:40 poikilotherm
Yeah, me too... ;-) I liked the concept
14:40 poikilotherm
But these days it seems to slow down things in complicated setups as dataverse
14:40 pdurbin
I even wrote about how I got religion about Vagrant at http://wiki.greptilian.com/vagrant
14:41 poikilotherm
Oh forgive me, your highness! I didn't want to mess with your tempels...
14:41 pdurbin
heh
14:41 poikilotherm
Still waiting for the vagrant up...
14:42 pdurbin
I created this repo just before taking the Dataverse job: https://github.com/dvn/dvn-install-demo
14:42 pdurbin
Because I was like "how the f do I install this thing?" :)
14:43 Jim_ joined #dataverse
14:43 poikilotherm
Hah! Done... Took me 16:52
14:43 poikilotherm
(minutes:seconds)
14:47 pdurbin
poikilotherm: oh, it's because you're downloading the base box. It'll be faster next time.
14:47 poikilotherm
I don't think I'll repeat this with Payara 5 :-D
14:47 poikilotherm
Hmm ok, I'll think over it. But tomorrow. Gotta go for today
14:48 pameyer
vagrant up usually runs me ~15m - when I've got the base box
14:48 pameyer
different vagrant setup though
14:48 pdurbin
pameyer: almost as slow as our ec2 script :)
14:48 Jim_
@pdurbin - sorry for breaking the build! I'll submit a PR for #5186 and look at #5185...
14:48 poikilotherm
Most of the time seems to be used for Maven Deps
14:48 pdurbin
having the war file already built speeds things up
14:48 poikilotherm
See you tomorrow guys
14:49 pdurbin
poikilotherm: bye!
14:49 Jim_
(and I thought I was doing so well - I even added new tests just for you in #5183 :-) )
14:49 pdurbin
Jim_: you're doing great! But thanks for looking at https://github.com/IQSS/dataverse/issues/5185 . Please keep me posted!
14:57 juancorr joined #dataverse
14:58 donsizemore
@pdurbin the only step that's particularly slow for me here is downloading solr, and i've resisted the urge to hard-code a link to ibiblio's apache mirror
15:00 pdurbin
donsizemore: good to know. I appreciate that you have a Vagrantfile in dataverse-ansible. It looks like a Dockerfile was contributed under "tests".
15:00 pameyer
@donsizemore - that's part of the reason I've been doing things like "check to see if it's already downloaded" before downloading glassfish/solr/etc
15:01 donsizemore
i have a Dockerfile?
15:01 pdurbin
heh
15:01 donsizemore
so i do! i'll poke at that next
15:01 pdurbin
This reminds me of last week's Ansible vs. Perl debate. :)
15:02 donsizemore
@pdurbin don't remind me.
15:02 pdurbin
donsizemore: oh, your doi variables pull request has merge conflicts. Can you please take a look? Or do you want me to try to resolve them?
15:04 donsizemore
yeah, the first PR was already in develop, so i pulled the two pertinent files into my branch. i may just need to do a fetch
15:04 pdurbin
I'm not sure. Git is weird.
15:05 donsizemore
@pdurbin having been a bystander to several git bombs i'm routinely grateful to do everything through pull requests
15:07 pdurbin
same
15:08 pdurbin
Jim_: thanks for the pull request for the SQL script. Do you think I should add a commit or two about process? Where to put the statements? That we love it when people email dataverse-dev after a merge? Or I could wait on this and move your pull request to QA.
15:30 pdurbin
At standup Danny said to go ahead and add something to the dev guide so I will.
15:30 pdurbin
juancorr: thanks for the comment at https://github.com/IQSS/dataverse/issues/4842#issuecomment-429896821 but were there supposed to be some screenshots?
15:47 pdurbin
pameyer: fingers crossed that I found a fix
15:48 Jim_
Phil - sounds good. FWIW, it was the existence of /scripts/database/3561-update.sql that gave me the idea of putting my sql where/how I did. Is that an obsolete file or is there a class of sql that is not part of the version upgrades?
15:49 pdurbin
Jim_: I was wondering about that too. Perhaps it's referenced in some release notes? If not, if it's just cruft, we should delete it. It's confusing.
15:52 Jim_
https://groups.google.com/forum/#!topic/dataverse-dev/puwkXiM4Tss is the only thing I see
15:53 pdurbin
Huh, I posted that. :)
15:54 Jim_
also re: 5185 - I don't see how my changes could have caused this - I didn't touch the ddi code directly and the export updates were just to use streaming and update the mimetypes. It could still be, but it's odd.
15:55 pdurbin
Yeah, very odd.
15:56 Jim_
Should this test have been running all along? We run tests in qdr and I haven't seen any error, but that's only through 4.9.4 (with qdr customizations). If I update the branch where I've been testing dpn to be up with dev and the failure appears, it would narrow it down...
15:56 pdurbin
Jim_: it looks like that one off sql script is mentioned in the 4.8 release notes: https://github.com/IQSS/dataverse/releases/tag/v4.8 . Not sure why we did things differently for one release.
15:58 pameyer
pdurbin: great
15:59 pdurbin
pameyer: I was wrong. Still broken. :/
15:59 Jim_
Ah - good catch - (I wonder why my google search for 'dataverse "3561-update.sql " ' didn't find the release page...)
16:00 pdurbin
Jim_: to support excluding email we have to pass SettingsServiceBean around. To check for ":ExcludeEmailFromExport".
16:00 pdurbin
And I'm sorry for having two conversations at once.
16:02 Jim_
no worries - how does your SettingsServiceBean relate - does that mean the test wouldn't normally run/give an error, or?
16:02 pameyer
Jim_: if you've only been running unit tests, you would have missed DatasetsIT
16:03 Jim_
ah - OK, it's all automated in jenkins, so I've never checked what actually runs...
16:03 pdurbin
Jim_ pameyer please take a look at https://github.com/IQSS/dataverse/pull/5047/files#r225223995
16:04 pdurbin
about new JsonPrinter(settingsService) being deleted
16:07 Jim_
got it - just read your comment.
16:07 pameyer
seems like it would make sense
16:10 Jim_
except that the method being called is static...
16:12 Jim_
I guess once you create one JsonPrinter with the settingservicebean, it is a static variable and available in subsequent static method calls...
16:14 pdurbin
Putting the code back to how it was isn't fixing it. Jim_ , can you please try?
16:17 Jim_
did you catch the analogous change at line ~157?
16:20 pdurbin
Jim_: ah ha! Thanks! Fixed! Pull request coming.
16:24 pdurbin
pameyer: https://github.com/IQSS/dataverse/pull/5191 if you want to try it
16:32 pameyer
pdurbin: had to kick docker, but should be running now
16:34 pdurbin
It'll be way faster than Vagrant, right? :)
16:35 isullivan joined #dataverse
16:35 pameyer
depends how many times I need to kick it... ;)
16:35 pdurbin
heh
16:40 jri joined #dataverse
16:47 pameyer
yup - looks good
16:55 donsizemore joined #dataverse
17:33 donsizemore joined #dataverse
17:57 donsizemore
@pdurbin i think i have #5173 in good shape (in both repos) but i'm getting warfile build errors due to repository.primefaces.org DNS
17:59 pdurbin
donsizemore: cool but I'm still seeing "Conflicting files" at the bottom of https://github.com/IQSS/dataverse/pull/5188 and the merge button is greyed out.
18:03 donsizemore
@pdurbin by "in good shape" i meant the branch was doing the needful =) will look at merge conflicts next
18:03 pameyer
@donsizemore DNS from aws? both network locations I'm hooked up to at the moment look ok for resolving repository.primefaces.org
18:06 donsizemore
i'm re-running the playbook now. it died during the warfile build step
20:33 pdurbin
Jim_: hi! I have a question about the getSpatialCoverages method you worked on. I think I may have found a bug having to do with "country".
20:54 pdurbin
I left a reminder to myself in the whiteboard drawing at https://github.com/IQSS/dataverse/issues/4371#issuecomment-430007532
20:55 pdurbin
pameyer: my quick feedback on https://github.com/IQSS/dataverse/pull/5182 is "why not change the password 'admin' to a variable in setup-all.sh instead?" but hopefully we'll get a chance to talk about this tomorrow. I gotta run.
21:02 pameyer
pdurbin: hopefully we can talk tomorrow - but short version is to make the smallest change possible. if anybody other than me reprovisions enough for it to be a problem, it might be worth revisiting.
21:03 pameyer
and if I should've replied somewhere in github, let me know - I missed it
21:16 jri joined #dataverse
21:43 pameyer joined #dataverse
21:52 Jim_
pdurbin - I'm back - what's your question...
22:04 pameyer left #dataverse
