Time
S
Nick
Message
03:04 jri joined #dataverse
06:04 jri joined #dataverse
08:20 stefankasberger joined #dataverse
10:08 stefankasberger joined #dataverse
10:27 pdurbin
stefankasberger: hi! Again, great presentation yesterday! I took some notes while you were talking. Do you think you could look them over? I'd like to send them out today.
10:50 jri joined #dataverse
11:10 Richard_Valdivia joined #dataverse
11:11 stefankasberger
sure, send me the link.
11:48 pdurbin
stefankasberger: https://docs.google.com/document/d/1nxvurGyVzxYKna-h0Qzl7emk6RoHELFFlwdPw7MESnI/edit
11:48 pdurbin
Richard_Valdivia: hi! I left you a message late yesterday: http://irclog.iq.harvard.edu/dataverse/2019-05-07#i_92678
12:02 stefankasberger
Have updated some minor parts. Thanks for the transcript.
12:49 donsizemore joined #dataverse
13:07 pdurbin
stefankasberger: thanks! Looks great. Also, did you see that for hacking we switched from a spreadsheet to a doc?
13:07 pdurbin
donsizemore: mornin'
13:08 poikilotherm joined #dataverse
13:08 Richard_Valdivia joined #dataverse
13:12 donsizemore
@pdurbin how are you sir
13:12 pdurbin
donsizemore: good. I've been thinking about chat protocols. Have you ever run an IRC or XMPP server?
13:13 poikilotherm
Morning guys
13:13 pdurbin
poikilotherm: good afternoon :)
13:15 donsizemore
@pdurbin i have not
13:15 poikilotherm
How are things going?
13:16 Richard_Valdivia
Hi pdurbin!!! Tks to send the log of yesterday conversation. We have problens with eletrical installation in our building and network goes down every time!!! Well. Of course is greath your idea work with your metadata expert. Tks by interest. I just talk with our librarians e give a feedback later...
13:16 poikilotherm
It's rainy here, but at least my presentation yesterday has been a success.
13:17 donsizemore
i'm imagining red hat's new container product had to have been designed in bahstin https://www.redhat.com/en/blog/daemon-haunted-container-world-no-longer-introducing-buildah-10
13:17 Richard_Valdivia
Morning for everyone!!
13:17 pdurbin
donsizemore: ok, in a previous life I helped run an openfire server.
13:18 poikilotherm
@donsizemore just this morning I looked into podman and buildah for building K8s images ;-)
13:18 pdurbin
Richard_Valdivia: perfect. Thank you. Our metadata expert is interesting in hearing what metadata standards are being used in bioinformatics, neurology, and marine biology. I *think* those are the disciplines/fields you mentioned. :)
13:19 pdurbin
poikilotherm: oh? Any slides? :)
13:19 poikilotherm
Yeah but in German
13:19 pdurbin
bah
13:19 poikilotherm
Sry
13:19 poikilotherm
Has been done for my colleagues
13:19 pdurbin
sure
13:20 pdurbin
donsizemore: heh. "Buildah".
13:21 pdurbin
Maybe we should all take a field trip to the secret speakeasy at the Boston Red Hat office after hacking a bit before the community meeting. donsizemore stefankasberger
13:21 poikilotherm
:'-(
13:22 pdurbin
poikilotherm: will Giacomo fund your travel?
13:22 pdurbin
stefankasberger: that reminds me, can you please ask Slava to join us in here sometime? :)
13:22 poikilotherm
Err... Haven't asked him about that...
13:22 pdurbin
hackers gotta hack
13:23 poikilotherm
Do you suggest having a hackathon at Dataverse Community Meeting?
13:23 donsizemore
@pdurbin why are you disappointed that the slides are in german?
13:24 donsizemore
@poikilotherm there's already something of a hackathon being planned via spreadsheet
13:24 pdurbin
poikilotherm: yes, there's a doc of ideas for hacking linked from the agenda.
13:24 donsizemore
@poikilotherm https://docs.google.com/document/d/19y2H_3fvHmni56JDucOHIYtYFbZDuZ-b2p37XvKdvjg/edit
13:25 donsizemore
@poikilotherm i'm really just going early to go eat at Dumpling House
13:30 poikilotherm
Ok, I could add to that agenda easily :-D
13:30 poikilotherm
But I dunno if this would work out.
13:30 pdurbin
Might be worth asking. :)
13:31 poikilotherm
I'll drop him an email
13:31 pdurbin
great, thanks
13:31 pdurbin
I feel bad that I haven't replied on the thread with you and MrK. :(
13:32 pdurbin
donsizemore: I'm happy to talk more about https://github.com/IQSS/dataverse-jenkins/issues/4 when you have time. :)
13:33 donsizemore
@pdurbin so you want to fire off docker-aio on develop build?
13:36 pdurbin
donsizemore: well, I don't want to over-specify what should be done. What I want is a replacement for how old Jenkins runs the API test suite on the phoenix server. Here's an example run. Please note that all the tests end with "IT" for "integration test": https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-develop/edu.harvard.iq$dataverse/387/testReport/edu.harvard.iq.dataverse.api/
13:37 pdurbin
Right now the API test suite (integration test suite) is run over the wire from the old Jenkins server to the phoenix server. This is not a requirement, however. Docker is fine. Whatever is fine.
13:37 pdurbin
Does that make sense? :)
13:39 donsizemore
in theory... your docker run-test-suite is what i snarfed for the ansible role. for jenkins we probably want this in docker
13:39 pdurbin
I don't think we have docker on old jenkins. Sounds fancy. Whatever works. :)
13:40 pdurbin
The main goal at this point is to catch regressions in the develop branch before we cut a release. This is the phoenix server's purpose in life. :)
13:45 pdurbin
Oh, and I guess a second "build passing" button at the bottom of our README. If we have two buttons maybe one could say "unit tests passing" and the other one could say "integration tests" passing. I don't know.
13:48 donsizemore
@pdurbin on running an IRC server. i vote for community slack
13:49 pdurbin
donsizemore: do I dare link you to my new "Searchable Linkable Open Public Indexed (SLOPI) Communication" proposal? :)
13:49 donsizemore
@pdurbin information wants to be free
13:50 pdurbin
pronounced "sloppy"
13:50 pdurbin
I was thinking. I wonder if I could write a paper about it in The Journal of Open Source Software. https://joss.theoj.org . Good idea? Bad idea? :)
13:55 pdurbin
probably the wrong journal
14:02 poikilotherm
Phil: send Giacomo an email asking for funding.
14:02 poikilotherm
Err
14:02 poikilotherm
-d +t
14:04 pdurbin
nice
14:04 pdurbin
It would be great to have you here.
14:05 poikilotherm
Well see... Lots of barriers ahead
14:05 * pdurbin
hands poikilotherm a jetpack
14:43 jri joined #dataverse
15:13 donsizemore
@pdurbin so what i'm hearing is i either want to set up a static VM for api tests from jenkins, or resume tinkering with docker-aio, or... possibly attempt to automate dataverse-k8s?
15:15 pdurbin
Well, phoenix is a static VM but maybe it could be ephemeral VM? Launched with the ec2 create script and terminated at the end? Or a different approach would be docker-aio. dataverse-k8s sounds fancy but complicated.
15:44 jri joined #dataverse
16:45 jri joined #dataverse
16:50 donsizemore joined #dataverse
17:46 jri joined #dataverse
18:20 donsizemore
@pdurbin i've been revisiting docker-aio. i can use it to bring up dataverse-develop, run-test-suite from the jenkins workspace dataverse repo, then publish the results. this seems pretty straightforward
18:46 donsizemore
@pdurbin and speaking of, in my testing the test suite in docker-aio doesn't complete. junit.framework.AssertionFailedError: expected:<200> but was:<500> then things hang
19:10 pdurbin
donsizemore: when you say "publish the results" do you mean the Jenkins will show AdminIT and friends?
19:10 donsizemore
@pdurbin one way or another?
19:10 pdurbin
:)
19:10 pdurbin
I'm not sure I follow you.
19:11 pdurbin
I've been having trouble with docker-aio lately too. Let me go try it on develop.
19:14 donsizemore
i just blamed myself per usual
19:14 pdurbin
Error processing tar file(exit status 1): write /tmp/dv/deps/solr-7.3.0dv.tgz: no space left on device
19:15 pdurbin
that... doesn't look good... but my laptop has plenty of free space
19:17 pdurbin
du -sh ~/Library/Containers/com.docker.docker shows 64 GB . Is that bad? I'm looking at https://forums.docker.com/t/no-space-left-on-device-error/10894
19:18 pdurbin
this by itself is 64GB: /Users/pdurbin/Library/Containers/com.docker.docker/Data/vms/0/Docker.qcow2
19:24 pdurbin
Huh. I moved that qcow2 aside and restarted Docker and now a new file is there that's only 1.3GB.
19:25 donsizemore
@pdurbin mine is nearly 2.2GB
19:26 pdurbin
You're on a Mac too?
19:31 pdurbin
much better, running the api test suite now
19:37 pdurbin
Hmm. Expected status code <200> doesn't match actual status code <400> at edu.harvard.iq.dataverse.api.SwordIT.testCreateDatasetPublishDestroy(SwordIT.java:558 as of a9605376c
19:37 pdurbin
But you said you're seeing a 500 error.
19:38 pdurbin
pulling the latest and trying again
19:47 donsizemore joined #dataverse
19:47 donsizemore
@pdurbin i got the 500s creating random users, on develop. everything else succeeded
19:48 donsizemore
@pdurbin they may cause a storm cloud in jenkins
19:48 pdurbin
I don't think anyone is looking at new jenkins but us. :)
19:49 donsizemore
okay =) well, i could get things that far in jenkins if you want, throw in a script to tear down the container. then iron out the test results page
19:50 pdurbin
sounds like progress toward the test results page I want :)
19:50 pdurbin
easier than ec2?
19:52 pdurbin
Huh, now I'm getting MoveIT.testMoveLinkedDataset:240 Expected status code <200> doesn't match actual status code <500>. On 656f3072b.
19:53 pdurbin
That was on docker-aio on my Mac I mean.
19:54 pdurbin
Using the some commit on phoenix it says the tests are passing: https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-develop/edu.harvard.iq$dataverse/388/testReport/edu.harvard.iq.dataverse.api/
19:58 pameyer joined #dataverse
19:59 pameyer
pdurbin: sounds to me like you've got a bunch of old images or partial images
19:59 pameyer
docker images ; and a bunch of docker rmi $foo should get that sorted out
20:00 pameyer
I don't have a Docker.qcow2; but that might be because of the vm type I'm using
20:00 pdurbin
pameyer: thanks! I moved the qcow2 file and I'm back in business. From the Docker perspective anyway. :)
20:00 pameyer
great!
20:00 pdurbin
500 error from Internal Exception: org.postgresql.util.PSQLException: ERROR: deadlock detected
20:00 pameyer
for what it's worth, I usually setup jenkins/docker pipelines to always remove the images when I'm done
20:01 pameyer
so far haven't run into space issues, but I also haven't been watching it
20:01 pdurbin
cool
20:01 pdurbin
Don is working on running the api test suite from docker-aio from new Jenkins.
20:03 pameyer
great
20:10 pdurbin
pameyer: thanks again for creating it!
20:18 pameyer
pdurbin: it's a lot of copy/paste from phoenix ;)
20:18 pameyer
glad it's useful
20:24 pdurbin
I always have to google for stuff like "how to ssh into docker" and "how to copy files from docker" but maybe someday it'll become muscle memory. :)
21:06 pameyer
pdurbin: that's what I keep cheetsheets (or notes, if you want it to sound better) for
23:33 jri joined #dataverse
