Time
S
Nick
Message
01:11 jri joined #dataverse
03:11 jri joined #dataverse
04:10 jri joined #dataverse
06:41 juancorr joined #dataverse
07:14 jonas4711 joined #dataverse
08:03 poikilotherm joined #dataverse
10:11 pdurbin
poikilotherm: morning. Feeling better?
10:11 poikilotherm
Yeah :-)
10:12 pdurbin
Great! Did you see the default=root pull request got merged? And that I made another one? :)
10:13 poikilotherm
Metamizole to the resuce
10:13 poikilotherm
Yes, I saw that.
10:13 poikilotherm
Crafting an issue currently
10:13 poikilotherm
So this PR has an issue to connect to :-d
10:14 poikilotherm
I would plege for not merging the if construct, but try to improve things.
10:15 pdurbin
Ah, you might make an alternative pull request?
10:15 poikilotherm
Will mention some ideas in my issue, and your solution is obviously one option
10:28 poikilotherm
Alright, telco done. Back to issues.
10:45 pdurbin
great
11:17 poikilotherm
Jesus.
11:17 poikilotherm
This grew long
11:17 poikilotherm
https://github.com/IQSS/dataverse/issues/5943
11:24 * pdurbin
scrolls
11:24 pdurbin
It's complicated. :)
11:24 poikilotherm
Yes
11:24 poikilotherm
But there is no easy answer
11:24 poikilotherm
And the problem itself is complex
11:25 poikilotherm
I tried to explain it briefly, hope you guys can follow my crude logic
11:26 pdurbin
You don't care if I close #5907 in favor of this new one, do you?
11:26 poikilotherm
Sure, go ahead.
11:26 poikilotherm
I linked it in my "header"
11:27 pdurbin
Yeah, I'm still reading. I saw. Thanks.
11:28 pdurbin
When you say "devs worked around it" what time frame are you talking about? This week? Recent pull requests from me?
11:30 poikilotherm
No.
11:30 poikilotherm
Ever since 2014
11:30 poikilotherm
Looks like you guys never heard of this bug.
11:31 poikilotherm
And you used to write code and it "just worked"
11:31 poikilotherm
Nobody stumbled over a case, where it didn't
11:31 poikilotherm
And as the GF versions was frozen since that time, you never had breaking code
11:32 poikilotherm
One thing lead to another, people got used to the way it worked
11:32 pdurbin
We certainly didn't know that javax.el-3.0.1-b03 was buggy. Do we have a link to the bug that's in there?
11:32 poikilotherm
I'm not blaming anyone, this just has been bad luck
11:33 poikilotherm
I didn't link the bugs, but I linked the madness article
11:33 poikilotherm
Lots of links and more in depth explanation in there
11:33 poikilotherm
Plus links to bug reports
11:34 pdurbin
Right. Let me see if I can find that specific bug report quick.
11:35 poikilotherm
You might be stopped by java.net being closed down
11:35 poikilotherm
Those things got moved to GitHub IIRC due to moving to Eclipse
11:35 pdurbin
Meanwhile, it sounds like you're ok with my option b style pull request: https://github.com/IQSS/dataverse/pull/5940 . Is that right? If it gets merged, you're ok with that?
11:37 poikilotherm
I would prefer not using an if. I don't it's very readable right now, as it is left unclear, why this is in there. And it is not easily testable
11:38 poikilotherm
Maybe a Math.max(1,page) would do the trick without code branch?
11:39 poikilotherm
Or a Math.max(0,...) in the paginationStart calculus
11:40 poikilotherm
Or even do not allow a value of 0 in the setter
11:40 poikilotherm
The var is assigned by JSF calling setPage()
11:41 poikilotherm
Maybe handling this in the setter with a max() is best. This is a enclosed things, which is easily testable
11:42 poikilotherm
Wow, that sentence got mixed up a lot...
11:42 poikilotherm
(Hope you get my point)
11:51 pdurbin_m joined #dataverse
11:51 pdurbin_m
I got recruited to carry a cello to school.
11:51 poikilotherm
Hehe. Good luck
11:55 pdurbin_m
in the rain
12:01 poikilotherm
Hope you have a good case to carry it safely ;-)
12:11 pdurbin
It's a soft case but mission accomplished. Just a drizzle, really.
12:11 pdurbin
poikilotherm: are you interested in making an alternative pull request?
12:15 donsizemore joined #dataverse
12:18 poikilotherm
If you want me to :-D
12:24 pdurbin
poikilotherm: sure!
12:40 pdurbin
poikilotherm: I'm seeing "This is fixed in EL org.glassfish:javax.el 3.0.1-b05 which is being pulled into wildfly 8.x" at https://github.com/eclipse-ee4j/mojarra/issues/3075#issuecomment-381966695
12:40 * poikilotherm
reads the bug repot
12:42 poikilotherm
Yeah, this is at least the perfect bug report for string madness :-)
12:42 poikilotherm
Congrats!
12:42 poikilotherm
Shall I add this to the description? (Dunno if you can add this)
12:42 poikilotherm
You did well, not so easy to dig in GitHub
12:49 pdurbin
Thanks. And this release has the bug fix, right? https://github.com/javaee/uel-ri/releases/tag/javax.el-3.0.1-b05
12:52 pdurbin
If so, one of these commits should have the bug fix on the el (expression language) side: https://github.com/javaee/uel-ri/commits/b2119f4523a307827a057ecf160fc490dac5f6f6
12:54 pdurbin
poikilotherm: you're saying that a title of " javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL doesn't work at all #3075 " makes sense, right? This is the behavior we have with Glassfish 4.1? That setting doesn't work?
12:57 poikilotherm
Lemme see. Is there a default value for the root dataverse, too?
12:57 jri joined #dataverse
12:59 poikilotherm
Hmm doesn't look like.
12:59 poikilotherm
Why has this been working in the past, then?
12:59 poikilotherm
Because this should be the same problem
13:00 pdurbin
It's all very, very confusing.
13:01 poikilotherm
It's like surgery
13:02 poikilotherm
Don't get afraid of the blood ;-)
13:03 pdurbin
Well, I'm not even sure I have the story straight. You wrote about princes and swords, plowshares and dwarfs. Elves. Should we restart for the 100th time what is going on?
13:03 pdurbin
Or should I think about it some more while I bike to work?
13:03 pdurbin
restate*
13:04 poikilotherm
Hehehe, I thought about going through JSF cycle, but then thought that would be too much
13:05 poikilotherm
Shall I refactor the description to squezze out humoristic things?
13:06 pdurbin
It sounds like our intention with the very first commit of Dataverse 4.0 that we set INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL to true for a reason. When you fill in a form, we don't want to save empty strings to our database. This was our intention. Is that right?
13:06 poikilotherm
Sounds right
13:07 poikilotherm
That is why this settings had been created upstream in the first place
13:08 poikilotherm
See http://balusc.omnifaces.org/2015/10/the-empty-string-madness.html at part "JSF 1.0/1.1"
13:09 poikilotherm
People needed to add their converter, which was clumsy
13:10 pdurbin
Ok. But you're saying we never noticed that this setting doesn't even work?
13:11 poikilotherm
Well it worked, but stopped working when EL 3.0 was released
13:11 poikilotherm
https://stackoverflow.com/questions/21880017/javax-faces-interpret-empty-string-submitted-values-as-null-does-not-work-anymor
13:12 pdurbin
Ok, please slow down. Did the setting work for us when we shipped Dataverse 4.0 on Glassfish 4.1 or not?
13:13 poikilotherm
I am not sure. I don't know what version of GF you used in those days
13:14 pdurbin
"Required Glassfish Versiion 4.1 is with weld v.2.2.4 module." http://guides.dataverse.org/en/4.0/installation/prerequisites.html ... we have always required Glassfish 4.1 for Dataverse 4.
13:14 poikilotherm
Dataverse 4.0 was released on 5/20/2015
13:15 pdurbin
"Required Glassfish Versiion 4.1 is with weld v.2.2.4 module." http://guides.dataverse.org/en/4.0/installation/prerequisites.html ... we have always required Glassfish 4.1 for Dataverse 4.
13:15 poikilotherm
Well, then this must have been broken all time, because the first fixed version is 3.0.1-b05, which is (at least when looking at the MANIFEST) not shipped with your GF version
13:15 poikilotherm
There might be a chance that there was some kind of backport
13:15 pdurbin
Ok, so let me ask my question again.
13:16 pdurbin
Did the INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true setting work for us when we shipped Dataverse 4.0 on Glassfish 4.1 or not?
13:17 poikilotherm
No, it should not have been working. :-/
13:17 pdurbin
Ok. Does this mean that all of our databases have empty strings in them that should have been null values? That we intended to be null values, I mean.
13:18 poikilotherm
Unsure. But we can take a look
13:19 poikilotherm
In theory, this should be the case, as long as there are no other things reverting this
13:19 poikilotherm
EclipseLink or other might do some magic
13:19 poikilotherm
Well we could dig deeper by add debugging to JSF phases
13:24 pdurbin
We could create a built in user and not fill in "affiliation" (not required) and see what gets saved to the database, a null or an empty string.
13:26 poikilotherm
Hmm. Wait a sec. Looking at DataversePage.init() now
13:27 poikilotherm
We know this works pretty well
13:27 poikilotherm
(On GF 4.1)
13:27 poikilotherm
When you access the page, I am sure that dataverseService.findRootDataverse() is used to get the root dataverse
13:27 poikilotherm
I know because I fell across that a few times :-D
13:28 poikilotherm
This means, going from that call up the branches, that alias and if have to be null
13:29 poikilotherm
Otherwise the else branch was never reachable
13:29 poikilotherm
And going ever higher, the whole thing is only reachable if ownerId is also null
13:30 poikilotherm
So this HAS to work
13:30 pdurbin
But what about my "affiliation" test? What do you think? Any bets on if it will be null or empty string in the database when you create a user through JSF? :)
13:30 poikilotherm
Let me look at the code first
13:31 poikilotherm
Err... do you have the class name at hands for this?
13:32 pdurbin
I'm doing this: \pset null 'Ø'
13:33 pdurbin
and this: select useridentifier,affiliation from authenticateduser;
13:33 pdurbin
You can look at the code if you want but I'm talking about behavior. :)
13:34 poikilotherm
Yeah. But please keep in mind that EclipseLink might transform this again
13:34 pdurbin
Oh?
13:34 pdurbin
So you aren't taking any bets? :)
13:34 poikilotherm
I don't know the layers in between
13:35 poikilotherm
The form is processed, services called, DAO and DTO done
13:35 poikilotherm
Lots of places things could happen to that String object
13:35 poikilotherm
The model contains no annotations for this attribute
13:36 poikilotherm
AuthenticatedUser.java:L89
13:37 pdurbin
oooo, I think I just found something
13:38 pdurbin
poikilotherm: is this "if ( nonEmpty(inf.getAffiliation()) ) {" a smoking gun? https://github.com/IQSS/dataverse/blob/v4.14/src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java#L153
13:39 poikilotherm
Lemme see where this comes from
13:39 pdurbin
Is it an indication that INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true never worked for us and that we put that "if non empty" in there to prevent empty strings from being saved to the database?
13:40 pdurbin
If I take out that "if" will an empty string be saved to "affiliation"?
13:40 pdurbin
Spoiler: right now a null is saved, probably because of that "if".
13:43 poikilotherm
that nonEmpty() is coming from StringUtil:L40
13:43 poikilotherm
it catches both null and empty strings
13:43 poikilotherm
Well this is definitly worth a try
13:45 poikilotherm
I try to find that XHTML file with the user form
13:45 poikilotherm
With those mapping params
13:46 poikilotherm
Ah I think I got it...
13:46 poikilotherm
dataverseuser.xhtml:L622
13:48 poikilotherm
Yes, this <p:inputText> is dropped inside the UserDisplayInfo object
13:48 jri joined #dataverse
13:50 poikilotherm
This stuff doesn't seem to be tampered with along the way
13:52 pdurbin_m joined #dataverse
13:52 pdurbin_m
walking to a bike share station
13:52 poikilotherm
:-)
13:53 pdurbin_m
because my stupid new lock broke
13:53 poikilotherm
Meh
13:53 poikilotherm
Please keep in mind: I have about 1h left
13:53 pdurbin_m
My last bike lock lasted 20 years and my new one lasted 20 days.
13:54 poikilotherm
Things tend to get worse over time
13:54 pdurbin_m
Dataverse too? :)
13:55 poikilotherm
Sure
13:55 pdurbin_m
:(
13:55 poikilotherm
Everything
13:55 poikilotherm
Chaos is a state using less energy
13:56 poikilotherm
I once had a shirt as a kid...
13:56 pdurbin_m
I think I disagree. Dataverse 4 has matured a lot. The team too. And we have a more engaged community.
13:56 poikilotherm
Yes, that got better :-)
13:56 poikilotherm
But it needed a lot of energey
13:56 poikilotherm
And still needs it
13:56 pdurbin_m
I don't sleep much. :)
13:57 poikilotherm
:-D
13:57 poikilotherm
I know.
13:57 poikilotherm
Poor Phil, up at 6 or even 5 am to talk to some silly german guy :-D
13:57 poikilotherm
I repeat: whoever invented timezones is to be sued by me
14:01 pdurbin_m
:)
14:01 pdurbin_m
Should we just set INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL to false? It's more honest.
14:02 poikilotherm
Well, at least in theory it should not matter
14:03 poikilotherm
I think about simply adding some debugging statements and give it a whirl
14:03 pdurbin_m
No. It does matter. It's an expression of behavior.
14:03 poikilotherm
But you want this setting to be set to true for later...
14:04 pdurbin_m
No. Set it to true in the future. When we are off Glassfish 4. And remove as many "if empty" lines as we can then. Right?
14:06 pdurbin_m
Make the current behavior explicit, I'm saying. Set it to false. Small chunks.
14:06 pdurbin_m
Step 1. Set to false.
14:06 pdurbin_m
Step 2. Switch to Payara 5.
14:06 poikilotherm
We definitly should have a look at verifying our theory first...
14:07 poikilotherm
It might be all wrong
14:07 pdurbin_m
Step 3. Set it to true and remove "if empty" code we don't need anymore.
14:07 poikilotherm
I am no expert in this stuff, maybe I dug at the wrong X
14:07 pdurbin_m
Do you like this plan?
14:07 poikilotherm
Sounds to easy :-D
14:07 poikilotherm
And I don't think it reflects all cases
14:08 poikilotherm
Like that converting to 0 with primitive int
14:26 pdurbin_m
poikilotherm: are you sure?
14:27 poikilotherm
Not yet
14:28 poikilotherm
Ok, I just enabled my Minikube cluster on our OpenStack cluster to be used locally with Telepresence
14:28 poikilotherm
In theory, I should be able to swap the dataverse container to a local one now :-)
14:28 poikilotherm
Then I can do development
14:29 poikilotherm
Running this stuff locally eats my RAM
14:36 pdurbin
poikilotherm: not sure yet but are you willing to help confirm your assertion?
14:36 poikilotherm
Obviously
14:37 pdurbin
Great!
14:37 poikilotherm
I am setting up gear to have test deployments running
14:37 pdurbin
Ah, great. Need any help?
14:38 poikilotherm
I don't think you can help me with telepresence or K8s ;-)
14:41 pdurbin
I barely even know what they are. :)
14:46 poikilotherm
Ok, this needs more investigation
14:46 poikilotherm
I'm outta here for today.
14:46 poikilotherm
Pick up kids ;-)
14:47 poikilotherm
Do roofing
14:47 poikilotherm
Cu tomorrow
14:47 pdurbin
bye!
14:48 jri joined #dataverse
14:50 pdurbin
donsizemore: hi! So we'll do the docker cp in the jenkins job?
15:01 donsizemore
@pdurbin yes, i'll put it there. was thinking docker-aio/ would get other docker runs but jenkins is fine. i closed my PR
15:07 pdurbin
donsizemore: cool. Again, either way is fine with me. Thanks!
15:30 pdurbin
donsizemore: how often does the cron tmp cleaner run?
15:46 donsizemore
nightly @ 0105
15:46 donsizemore
we won't need it once we're either on jenkins permsies and/or don't use /tmp
15:49 pdurbin
donsizemore: well, I think Jenkins keeps building because of the webhook. Should I delete the webhook?
16:21 pdurbin
donsizemore: also, did you ever hear back from Andrey if he got the confirmation email or not?
17:16 donsizemore
he said danny wrote to him
17:16 donsizemore
i dunno much more than that
17:50 pdurbin
Hmm, ok. Thanks.
17:51 pdurbin
donsizemore: so should I delete the webhook? So files stop showing up in /tmp? Or maybe you can give me sudo access to run the tmp cleaner? I'm flexible. :)
17:55 donsizemore
i can just disable the job until you're done testing. i'm not "sposed" to give an outsider sudo
17:56 donsizemore
i deleted the trigger for now
17:56 pdurbin
donsizemore: thanks! Can you please delete the files in /tmp too?
18:00 donsizemore
done
18:03 pdurbin
Thanks!
18:22 donsizemore
sorry i'm still mired in backup cleanup. with large VMs this stuff takes forever
18:31 pdurbin
No problem. BUILD SUCCESS as of a9483ac just now for the API test suite.
18:32 pdurbin
donsizemore: when do I declare victory? How many BUILD SUCCESS in a row?
18:38 donsizemore
@pdurbin i got a different error each time, so i say a success is a success
18:39 pdurbin
:)
18:39 pdurbin
I'm thinking 3 in a row.
18:41 pdurbin
But I'm doing a `git pull` in between each one, which isn't very scientific of me. :)
18:47 donsizemore
it's what jenkins would do (and what i had been doing (and still failing each time))
18:48 pdurbin
True, it's a little more real world to have the pulls in there. :)
18:49 pdurbin
Uh, oh, 500 errors now. And the "run the api test suite" script slows way down. This happened yesterday too. Why?
18:49 * pdurbin
runs `top`
18:49 donsizemore
welcome to my world
18:49 pdurbin
heh
18:50 pdurbin
Do we need to clean out Docker extra hard between runs?
18:50 pdurbin
bjonnh: help!
18:59 donsizemore
i had been periodically removing old images
19:00 pdurbin
donsizemore: at some point I'm just going to have to hit Ctrl-C on this 'run the tests' script, I think. Any idea why Glassfish is so unhappy in the docker container?
19:01 pdurbin
I guess I should be running `top` from inside the container rather than from the host, the Jenkins server.
19:03 pdurbin
The tests are very
19:03 pdurbin
slowly
19:03 pdurbin
failing
19:03 pdurbin
one
19:03 pdurbin
by
19:03 pdurbin
one
19:10 pdurbin
donsizemore: is it possible that the `docker rm` or whatever isn't working?
19:11 pdurbin
I'm asking because when I "ssh" into the container, an older commit than I expect is deployed to glassfish.
19:22 bjonnh
pdurbin: docker run --rm image
19:22 bjonnh
is pretty useful
19:22 bjonnh
if you want to run a container but not keep it after
19:22 bjonnh
there is a docker purge command to clean up
19:23 donsizemore
a day or two ago i removed all containers and images
19:23 bjonnh
docker system prune -f
19:23 bjonnh
is a way to clean hard
19:23 donsizemore
if you want me to kill it and wipe everything out
19:23 bjonnh
docker system prune -af
19:23 bjonnh
kills everything IIRC
19:23 donsizemore
@bjonnh millenial for "nuke" =)
19:24 bjonnh
docker X prune -f
19:24 bjonnh
where X can be image, network, volume
19:24 bjonnh
container maybe?
19:28 bjonnh
also be careful, docker likes to mess with firewalls, don't expose ports you don't want to REALLY expose
19:28 bjonnh
so many kubernetes clusters open to the world :D
19:29 donsizemore
PORT STATE SERVICE 8080/tcp closed http-proxy
19:30 donsizemore
oh, heh. @pdurbin killed the container
19:30 donsizemore
i thought we wanted to shut down and nuke the container once tests complete, hence the server.log docker-cp
19:59 sivoais joined #dataverse
20:44 pdurbin
Did I? I don't know what I'm doing. :)
