Time
S
Nick
Message
05:47 jri joined #dataverse
07:06 jri joined #dataverse
08:00 jri joined #dataverse
09:22 dataverse-user joined #dataverse
10:20 jri joined #dataverse
10:30 jri joined #dataverse
10:35 jri_ joined #dataverse
10:36 poikilotherm joined #dataverse
10:37 poikilotherm
Good morning America :-)
10:37 pdurbin
mornin
10:38 poikilotherm
We sunny weather at about 67°F
10:38 poikilotherm
+have
10:38 poikilotherm
Everything is nice and shiny
10:39 poikilotherm
Hope you guys enjoyed your vacation during the easter holidays
10:39 pdurbin
Nice. I don't have heat at home but the furnace guys are coming back today. Yes, nice Easter. I got to meet my new nephew, who is 7 months old.
10:39 jri joined #dataverse
10:41 poikilotherm
Uh, cold shower...
10:41 poikilotherm
Baby sounds nice :-)
10:41 pdurbin
Hot shower. Separate hot water tank. :)
10:42 pdurbin
and it only got down to 8 C (47 F) last night
10:43 poikilotherm
*thumbs up*
10:44 pdurbin
you may have noticed that the 4.13 artifacts have been uploaded
10:45 pdurbin
war file, etc
10:45 poikilotherm
Yeah, I did. Just merged #42
10:46 poikilotherm
Will tag as soon as the master build on Docker Hub is done.
10:48 pdurbin
nice
10:48 pdurbin
Any more thoughts on making Dataverse testable?
10:48 pdurbin
(more testable)
10:49 poikilotherm
It might be woth a try to get things going with k3s
10:49 poikilotherm
I am actually doing this in https://github.com/IQSS/dataverse-kubernetes/pull/43
10:49 poikilotherm
K3s is small enough to fit into a CI pipeline
10:50 poikilotherm
And it had been fast forward to use and deploy
10:50 poikilotherm
Altough waiting for the glassfish war deployment is still an issue :-/
10:51 poikilotherm
Its taking ages :-(
10:51 pdurbin
At Easter a friend asked me what k8s is and I told her what I told you about renaming a certain git repo. What's k3s?
10:53 pdurbin
"5 less than k8s"?
10:53 pdurbin
I miss searchbot.
10:53 poikilotherm
Yes!!!!
10:53 poikilotherm
:-D
10:53 poikilotherm
http://k3s.io
10:53 pdurbin
Here was searchbot in action: http://irclog.iq.harvard.edu/dataverse/2016-04-04#i_33685
10:55 poikilotherm
Well why dont you get him back?
10:55 poikilotherm
"Him" haaha
10:55 pdurbin
google shut down their api
10:55 pdurbin
or something
10:56 pdurbin
now iqlogbot is lonely, the only bot
10:56 poikilotherm
Meh
10:56 pdurbin
unless candy`_ is a bot...
10:57 pdurbin
sometimes it's hard to tell
10:58 pdurbin
Anyway, my guess is that you're oriented toward testing releases to prevent regressions from hitting your users. Basically, you'll skip deploying releases with bugs to prod. Is that right?
10:59 poikilotherm
For k3s?
10:59 poikilotherm
Sorry, I am puzzled about your current focus
11:00 pdurbin
Ah, well, I guess I've been thinking about the phoenix server. You're familiar with it, I assume.
11:01 pdurbin
I set it up years ago to run the API test suite before cutting a release.
11:01 poikilotherm
About your bots: you could use sth. like hubot framework and google custom search engine with API key
11:01 pdurbin
oh, maybe all I need is a key ok. searchbot was a supybot
11:01 poikilotherm
Yeah, I am familiar with Phoenix :-)
11:02 pdurbin
phoenix is an old centos 6 box in the basement of our building
11:02 poikilotherm
API keys are for free in most cases when used for low traffic
11:02 poikilotherm
Aye
11:02 poikilotherm
You mentioned that a few times ;-)
11:02 pdurbin
what I want, I think, is a new phoenix on ec2
11:02 pdurbin
and maybe phoenix can be ephemeral
11:03 pdurbin
only up for as long as the tests need to be run
11:03 pdurbin
what's important to me is if the api test suite is passing or failing
11:03 poikilotherm
Actually that's just the thing that happens in a CI pipeline
11:04 pdurbin
sure, more automated
11:04 poikilotherm
You spin up the services, run the tests and tear down
11:04 pdurbin
I've been clicking "build now" for, uh, four years?
11:04 pdurbin
381 times, I guess
11:04 poikilotherm
:-D
11:05 poikilotherm
Some people are using Jenkins or similar together with Github comments.
11:05 poikilotherm
On some projects, you will see sth. like /jenkins test
11:05 pdurbin
occasionally, someone else clicks "build now" but it's usually me
11:05 poikilotherm
They trigger a build in that way, because building every commit or PR would trigger way too much load
11:06 poikilotherm
There are definitly a lot of ways to trigger tings
11:06 pdurbin
I guess it's only been 3+ years since I set up phoenix: https://groups.google.com/d/msg/dataverse-community/X8OrRWbPimA/wcmQx5VZCQAJ
11:06 poikilotherm
You could setup a simple bot listing on IRC
11:06 poikilotherm
It's called ChatOps then :-D
11:06 pdurbin
phoenix has been great, it has caught many many regressions
11:07 poikilotherm
:-)
11:07 poikilotherm
I totally agree with that
11:07 pdurbin
but it's oriented toward preventing regressions from making it into a release
11:07 pdurbin
I make a lot of noise if we're about to cut a release and the api tests are failing.
11:07 pdurbin
I was actually really pleased that the release was delayed last week due to api test failures.
11:08 pdurbin
It means people are running the tests on phoenix even when I'm on vacation. :)
11:08 pdurbin
I'm going somewhere with all of this but it's taking me a while to get there.
11:09 pdurbin
Are you running the API test suite? In any of your k8s stuff?
11:09 poikilotherm
Not yet
11:09 poikilotherm
But I am really eager to try it
11:09 pdurbin
awesome
11:09 pdurbin
good answer :)
11:10 pdurbin
please let me know if I can help
11:10 poikilotherm
One could argue that my project which is kind of downstream should not run tests for upstream
11:10 poikilotherm
But I would like to give it a try
11:10 pdurbin
sure, but I'm hoping to learn from you
11:11 poikilotherm
Maybe I will just add it to a feature branch and never merge it in master
11:11 poikilotherm
Please be aware that I will stick with Travis for dataverse-kubernetes
11:11 poikilotherm
I tried a few Jenkins things and it has been tedious and slow to get things rolling
11:12 pdurbin
yeah, I saw you're sticking with Travis, that's fine
11:12 poikilotherm
This is not due to bad hardware or sth. like that. IMHO this is just Jenkins.
11:12 pdurbin
it is what it is
11:12 pdurbin
maybe in the future we'll all be on gitlab and gitlab-ci, who knows
11:13 poikilotherm
Yeah. Pipelines are a nightmare in Jenkins.
11:13 pdurbin
but for now, jenkins has some mind share
11:13 poikilotherm
Who uses Groovy?
11:13 pdurbin
heh
11:13 poikilotherm
YAML for Travis or Gitlab CI is much less painfull
11:13 pdurbin
as long as it works
11:13 poikilotherm
;-)
11:13 pdurbin
as long as we catch regressions
11:14 poikilotherm
And I enjoy the matrix feature of Travis.
11:14 poikilotherm
I just tagged dataverse-kubernetes v4.13
11:14 poikilotherm
Images are being build on Docker Hub
11:14 pdurbin
travis is great. I have it set up on a number of iqss projects
11:15 pdurbin
What happens before you tag? They just go to "latest"?
11:16 poikilotherm
Aye
11:16 pdurbin
huh, I don't see a 4.13 tag yet
11:19 poikilotherm
I should mentions CAP theorem here
11:20 poikilotherm
https://en.wikipedia.org/wiki/CAP_theorem
11:20 pdurbin
I've heard of it.
11:20 poikilotherm
Meaning: try "reload" ;-)
11:21 poikilotherm
Tag building on Docker hub is still on its way
11:21 poikilotherm
But you should see a tag on Github
11:21 pdurbin
ah, ok
11:21 pdurbin
I was looking at Docker Hub.
11:21 poikilotherm
I want to move image builds to Travis
11:22 poikilotherm
Faster, more reliable, more flexibility and more consistent to have tests and image builds under one UI
11:22 poikilotherm
And: scheduled builds
11:23 poikilotherm
Very important as the base images (CentOS) is updated frequently
11:23 pdurbin
And greater visibility into the build, right? I could just watch the Travis console that way.
11:23 poikilotherm
You have access to build logs as owner of the repo
11:23 poikilotherm
(at Docker hub)
11:23 pdurbin
Ok, so it isn't public like Travis.
11:24 pdurbin
bummer
11:24 poikilotherm
Btw. to use k8s for api tests BEFORE a release happens, you will need to modify the images
11:24 pdurbin
sure
11:24 poikilotherm
I thought about using my last tagged release and inherit fro mthat
11:24 poikilotherm
And just swap the WAR file at build time
11:25 poikilotherm
Push in the WAR file build from the dev code
11:25 pdurbin
I asked Don to make that happen in his dataverse-ansible world and he delivered. It has been fantastic: https://github.com/IQSS/dataverse-ansible/issues/22
11:25 poikilotherm
Obviously this needs to be tweaked when other things like the JSON files change
11:25 poikilotherm
Or Solr things change.
11:25 poikilotherm
But it would be a start
11:25 pdurbin
Sure.
11:26 pdurbin
On a related note, did you see Slava's slides about his CI pipeline?
11:26 poikilotherm
Err not sure
11:27 poikilotherm
Docker hub build is done :-)
11:27 pdurbin
I posted some screenshots at https://github.com/IQSS/dataverse/issues/5725#issuecomment-482083170
11:27 pdurbin
What do you think?
11:28 poikilotherm
Lots of good stuff in there
11:28 poikilotherm
I hope they get it into upstream
11:35 pdurbin
I asked him for an account: https://github.com/IQSS/dataverse/pull/5751#issuecomment-482349090
11:35 pdurbin
Do you want one?
11:38 poikilotherm
Dunno. I don't think I can look into it right now :-/ We need to get Dataverse in production first... ;-)
11:39 poikilotherm
I think we will focus later on more on developing the core further and enhance the images
11:39 poikilotherm
This should be benefitial for everyone :-D
11:39 poikilotherm
Thinking about creating a feature branch with a Payara 5 base image...
11:39 pdurbin
Sure. Right now you're mostly accepting the core as-is. You contributed to the S3 clone driver (thank you!!) but I don't think you're hacking on the core much these days.
11:40 poikilotherm
Yeah
11:40 poikilotherm
Focusing on service building right now
11:40 pdurbin
which is perfect
11:40 pdurbin
extremely valuable
11:40 pdurbin
thank you
11:41 poikilotherm
I think I will take some time now to write some more docs...
11:41 poikilotherm
"How does dataverse-kubernetes work"
11:41 poikilotherm
Maybe embedding a graph with https://g.gravizo.com/
11:43 pdurbin
my only open issue is https://github.com/IQSS/dataverse-kubernetes/issues/12 :)
11:45 poikilotherm
Well I could give it a try...
11:45 pdurbin
I'm happy to create an account for you under our AWS like I did for Don.
11:45 poikilotherm
When you give me some access credentials for an EC2 instance
11:46 poikilotherm
Ok, I give this a quick shot right now
11:46 poikilotherm
Let's see how easy this is
11:46 pdurbin
Do you want an account under your name or should it be a "service" account that https://jenkins.dataverse.org will use some day?
11:47 poikilotherm
Um maybe let me play with an account under my name first
11:47 poikilotherm
Services: later. When this stuff works
11:47 pdurbin
makes sense. your usual nick and work email, I assume
11:48 poikilotherm
YEs please
11:56 pdurbin
Ok, please check your email.
12:00 poikilotherm
Thx!
12:05 poikilotherm
I will need these permissions:
12:05 poikilotherm
AmazonEC2FullAccess
12:05 poikilotherm
AmazonRoute53FullAccess
12:05 poikilotherm
AmazonS3FullAccess
12:05 poikilotherm
IAMFullAccess
12:05 poikilotherm
AmazonVPCFullAccess
12:05 poikilotherm
I don't think I can see those in my account... At least I could not find them
12:07 pdurbin
I gave you "full" just like me.
12:07 pdurbin
Let me look again.
12:07 poikilotherm
Ok
12:07 poikilotherm
kops states that I should create a special user with limited permissions as show above...
12:08 poikilotherm
Can I create a user myself?
12:08 poikilotherm
Ah I think I just found it :-D
12:08 pdurbin
Oh, limited permissions. Well, I just created an account for you so you should be able to create a second one.
12:09 pdurbin
I would suggest testing your account by following http://guides.dataverse.org/en/4.13/developers/deployment.html
12:55 poikilotherm
Ok first lesson learned: kops uses K8s v1.11 by default. Upgrading to 1.13 took me about 24m resulting from a lot of waiting...
12:56 pdurbin
ah
12:56 pdurbin
I don't know what kops is.
12:56 poikilotherm
A little tool to install K8s on AWS/GKE/...
12:56 pdurbin
Ah, ok.
12:57 pdurbin
The furnace guys are here. Once I get this dishwasher loaded I'll bike to work.
12:57 poikilotherm
:-)
12:59 pdurbin
see you on the other side
13:00 poikilotherm
Cu :-)
13:16 donsizemore joined #dataverse
13:23 poikilotherm
Hey pdurbin, it "just worked"
13:24 poikilotherm
I'll write some docs about it...
13:24 poikilotherm
And commit some YAML files
13:51 pdurbin
poikilotherm: fantastic! Thanks!
13:51 pdurbin
donsizemore: ^^
14:09 poikilotherm
Oi, pdurbin
14:09 poikilotherm
https://github.com/IQSS/dataverse-kubernetes/blob/12-k8s-aws/docs/aws-kops.md
14:10 poikilotherm
Give it a shot :-D
14:10 poikilotherm
It worked like a breeze here... ;-)
14:11 poikilotherm
I'll be out for today. Feel free to leave feedback in the issue...
14:18 pdurbin
poikilotherm: thanks!!!
14:18 pdurbin
at the moment I'm in a design meeting discussing https://github.com/IQSS/dataverse-metrics/issues/11
14:19 donsizemore
@pdurbin is that a hint?
14:20 pdurbin
donsizemore: hmmm? :)
15:10 pdurbin
donsizemore: still there? Question about metrics.
15:10 donsizemore
yes sir
15:11 pdurbin
can you bump it up from 12 months to 36, please?
15:11 pdurbin
oh, and deploy the latest release?
15:12 donsizemore
yis
15:16 donsizemore
done and updating now
15:17 donsizemore
(and taking much longer to update, i might add)
15:26 donsizemore
um... https://dataversemetrics.odum.unc.edu/dataverse-metrics/
15:26 donsizemore
"num_months_to_process": 36 — maybe i need to nuke the cache and re-run?
15:29 donsizemore
the cache hierarchy now goes back 36 months beneath */toMonth/ but the... oh, it's my browser.
15:30 donsizemore
now we're back 36 months but we lost the "month" labels. mouse-over displays them
15:46 pdurbin
donsizemore: hi! Sorry, I was at standup. Are you done already?!? https://dataverse.org/metrics looks good!
15:48 pdurbin
And yes, we know that the month labels get squeezed out. We're planning to fix that in a future release.
16:41 donsizemore joined #dataverse
16:50 donsizemore
@pdurbin added https://data.inra.fr back in and re-running.
16:50 donsizemore
@pdurbin I had removed them because they kept throwing 503s
16:51 pdurbin
donsizemore: and speaking of which, on my box I removed scholars portal because I was getting this error: https://github.com/IQSS/dataverse-metrics/issues/12
17:37 jri joined #dataverse
18:03 donsizemore
@pdurbin did you still want me to try to reproduce #5722? it's closed now and i don't see the re-opened replacement (unless it's been closed as well)
18:56 pdurbin
donsizemore: hi! That actually relates to what I started talking about around this time yesterday... thoughts on jenkins etc.
19:02 donsizemore
and automated testing
19:04 pdurbin
right! so
19:04 pdurbin
so far we have one job and it's awesome: https://jenkins.dataverse.org/job/IQSS-dataverse-develop/
19:04 pdurbin
passing, even :)
19:05 pdurbin
but what's the next logical step?
19:05 pdurbin
that job does a mvn test and mvn package, right?
19:06 pdurbin
config.xml says it does this: mvn -DcompilerArgument=-Xlint:unchecked test -P all-unit-tests package
19:06 pdurbin
which seems great
19:06 pdurbin
but now what? deploy to ec2?
19:07 pdurbin
I was going to create an issue but I wanted to talk it out a bit first. Or I can go ahead. What do you think?
19:09 donsizemore
um, locally we have several jobs which just fire off ssh with a script as an argument.
19:10 donsizemore
authentication is just simple ssh-key, and in each case the script simply redeploys the warfile and restarts glassfish/payara
19:10 donsizemore
i do like the ec2 approach; we'd just give jenkins your aws credentials(?)
19:11 pdurbin
yep, I could create a service user or whatever
19:12 pdurbin
we would just call the ec2-create-instance.sh script from jenkins, right?
19:12 donsizemore
only potentially messy part would be automated teardown, assuming you wanted to examine the running VM first
19:12 donsizemore
"with AWS, you don't pay for what you use, you pay for what you forget to turn off"
19:12 pdurbin
heh
19:12 pdurbin
I mean, maybe we don't need ec2 yet? Maybe we can deploy to glassfish within docker on the jenkins server? I don't know.
19:13 pdurbin
but the next step is to deploy the war file somewhere, I think
19:15 donsizemore
i do like ec2 though that costs $$$
19:15 donsizemore
if that's untenable for whatever reason i can set up a test glassfish instance here
19:16 pdurbin
let's try ec2 first. I'd like that "create" script to be exercised regularly anyway. it's a good test of the master branch of dataverse-ansible
19:17 pdurbin
so does a second job get added to config.xml? Or do we create a new file called config2.xml or something?
19:17 donsizemore
is there a better way for me to do this than to give our jenkins user your ec2 credentials?
19:17 donsizemore
you can add any number of post-build actions in jenkins
19:18 pdurbin
I'm not sure of the best way to handle the credentials but I think we should think of both the jenkins server and the ec2 credentials as belonging to the dataverse community. :)
19:19 pdurbin
we all want the software to be tested
19:19 pdurbin
nobody wants regressions
19:19 donsizemore
unless i'm allowed to regress to kindergarden with afternoon naps on a towel, i agree.
19:20 pdurbin
:)
19:20 pdurbin
you have the same privs I do so you could create the service account if you want
19:20 donsizemore
we'll start with EC2 and if it sux in some way we can always change it
19:21 pdurbin
yeah, we're just experimenting
19:21 pdurbin
you saw Slava's CI pipeline, right? I posted some screenshots
19:21 donsizemore
yes
19:21 donsizemore
though that was last week so i'd have to dig them up
19:22 pdurbin
here they are: https://github.com/IQSS/dataverse/issues/5725#issuecomment-482083170
19:22 pdurbin
he's also using jenkins
19:23 pdurbin
so is qqmyers
19:23 pdurbin
so I feel like we an all learn from each other
19:23 pdurbin
can*
19:26 pdurbin
Have you seen how I have the phoenix stuff set up as 3 chained jobs? I'm not sure if I'm doing it right but it's been working ok, I guess. It catches bugs. :)
19:27 pdurbin
here are the 3 chained jobs: http://guides.dataverse.org/en/4.13/developers/testing.html#how-the-phoenix-tests-work
19:29 pdurbin
part of the problem is there's no transparency into how those jobs are configured
19:30 pdurbin
no config.xml to download
19:30 pdurbin
that's what https://github.com/IQSS/dataverse/issues/1936 was about
19:31 pdurbin
but in that issue I just posted screenshots because that's all I know how to do
19:36 donsizemore
i've been updating our config.xml in the dataverse-jenkins repo, and am happy to post the script i'll use to fire off ec2 - no secrets there
19:37 pdurbin
awesome
19:43 donsizemore
must run but hope to finish that up tomorrow morning? have a great afternoon
19:43 pdurbin
you too. thanks!
20:35 pdurbin joined #dataverse
21:49 jri joined #dataverse
