Time
S
Nick
Message
06:40 juancorr joined #dataverse
06:56 poikilotherm joined #dataverse
07:47 jri joined #dataverse
08:57 stefankasberger joined #dataverse
08:58 stefankasberger4 joined #dataverse
09:13 xarthisius joined #dataverse
09:13 xarthisius joined #dataverse
10:44 * pdurbin
plays The Heat Is On by Glenn Frey
10:57 * poikilotherm
plays Hearts on Fire by Hammerfall
10:58 poikilotherm
Morning pdurbin :-)
10:58 poikilotherm
Dunno if you saw https://github.com/IQSS/dataverse-kubernetes/issues/12#issuecomment-486569468
10:59 poikilotherm
It gave me some hard time so I lost motivation :-D
10:59 poikilotherm
(And deleted it, so it doesn't empty IQSS purses)
11:00 pdurbin
Right. Yes. I did read that. I guess I assumed you were using EKS until I read your readme last night. :)
11:00 poikilotherm
LOL
11:01 poikilotherm
Sry I should have mentioned that more explicitly
11:02 pdurbin
no worries
11:02 pdurbin
How many EC2 instances are spun up?
11:05 poikilotherm
Currently none
11:05 poikilotherm
I terminated them all
11:05 pdurbin
heh, I mean when it was running
11:05 poikilotherm
With kops or EKS?
11:05 pdurbin
kops
11:06 poikilotherm
IIRC kops uses 3 by default. I definitly had 3 nodes in the cluster
11:06 pdurbin
ok, so three ec2 instances. glassfish, solr, and postgres?
11:09 poikilotherm
Nope. 3 Kubernetes nodes, one acting as master
11:10 pdurbin
ok, obviously, I have a lot to learn :)
11:10 poikilotherm
You can restrict where which Pod/container is going to run, but most of the time this is not what you want :-D
11:10 poikilotherm
You might think of a worker roughly as "a server with Docker running on it"
11:10 pdurbin
ok, so they are called workers
11:11 poikilotherm
That's basically it. Obviously, Kubernetes adds more sugar, but the basic thing is the container engine running your containers
11:11 pdurbin
sure, orchestration
11:15 poikilotherm
Yeah. Those run in containers, too.
11:16 poikilotherm
I should also mention the overlay network, which enables communication across nodes/clusters
11:16 pdurbin
sounds like a lot is going on :)
11:17 poikilotherm
Of course :-D
11:17 poikilotherm
Setup using "old fashioned" is always lighter. At least as long as you are using a simple deployment with one server and you have nothing else and want no scaling etc.
11:18 pdurbin
Any thoughts on kops vs EKS? Is one better than the other? kops is lighter?
11:18 poikilotherm
With kops or similar tools, you have the complete cluster in your hands and your administration. Which also means you should know what you do.
11:19 poikilotherm
Using EKS takes away that burden more or less
11:19 poikilotherm
It gets even more into a "turnkey" solution
11:19 poikilotherm
When you want to combine public/private cloud, you will most certainly stick with things like kops
11:20 pdurbin
Sure, like how we use Amazon's "postgres as a service".
11:20 pdurbin
turnkey
11:20 pdurbin
proprietary
11:21 poikilotherm
For my testing yesterday, kops was up and running much faster than fiddling around with EKS
11:21 pdurbin
that's awesome
11:21 pdurbin
Do you think kops should be added to a future version of http://guides.dataverse.org/en/4.13/installation/prep.html#choose-your-own-installation-adventure ?
11:22 poikilotherm
Me being relatively unfamiliar with AWS was pretty happy that it "just worked", while I couldn't figure out why my EKS cluster did not work as expected
11:22 poikilotherm
Maybe we should instead point people to dataverse-kubernetes?
11:22 pdurbin
sure! want to make a pull request? :)
11:22 poikilotherm
I started docs over there for this flavor of deployment to keep it simpler to maintain
11:23 poikilotherm
This is still pretty much "in flow"
11:23 poikilotherm
Still learning here, too ;-)
11:23 poikilotherm
Sure
11:24 pdurbin
\o/
11:24 poikilotherm
Will be AFK for a while now. See you later.
11:25 pdurbin
Bye! Thanks for all your hard work on this.
11:26 poikilotherm
You're welcome :-)
11:38 jri joined #dataverse
11:50 pdurbin
I just attempted to summarize the situation (and other updates) at https://github.com/IQSS/dataverse/issues/4665#issuecomment-486639913
11:51 pdurbin
larsks: do you have anything to do with running Dataverse in Docker? I'm not sure which installation of Dataverse (if any), you're involved with.
12:20 donsizemore joined #dataverse
12:48 pdurbin
donsizemore: I forgot to say that everyone really appreciates you updating https://dataverse.org/metrics so quickly yesterday! Thanks!
12:49 donsizemore
@pdurbin ah do sumthin raht, wunst in a while
12:50 pdurbin
:)
12:50 donsizemore
i'm also comforted that @poikilotherm found EKS to be odiously finicky
12:50 pdurbin
yeah, now I need to circle back with folks here doing EKS (non Dataverse app, not open source yet)
12:51 pdurbin
stefankasberger4: that reminds me, are you around?
12:57 pdurbin
donsizemore: so what million other things have I asked you to do lately? Please let me know when enough is enough. :)
12:58 donsizemore
@pdurbin um, i'm plunking at the ec2-create bit for jenkins now, unless you'd rather use docker or an existing glassfish instance
12:59 pdurbin
ec2 create sounds perfect
13:00 pdurbin
exercises a lot of code
13:00 donsizemore
@pdurbin the output of ec2-create-instance.sh should be public in the job output... automating the instance destruction will be... the next step
13:00 donsizemore
@pdurbin do you want jenkins to send the AWS SSH key somewhere?
13:00 pdurbin
I don't see why not. A private key will be generated but it won't be downloadable, will it?
13:02 pdurbin
I won't need the private key that's generated if you add https://github.com/pdurbin.keys to /home/centos/.ssh/authorized_keys
13:05 donsizemore
i do dat
13:05 pdurbin
thanks!
13:06 pdurbin
out of curiosity, are you going to scp the war file over from jenkins or build it again on the ec2 instance?
13:09 donsizemore
eh, just build it again? each job gets triggered when develop is touched
13:12 pdurbin
ah, so they aren't chained, sounds fine
13:13 pdurbin
If the branch doesn't even compile, both jobs will fail. :)
13:45 donsizemore
well, i could modify ec2-create so that we only build the warfile once, but in theory the result should be the same
13:46 pdurbin
up to you
13:46 pdurbin
we are good at deferring work :)
13:48 larsks
pdurbin: I'm not *directly* involved with any dataverse instance.
13:48 pdurbin
just the storage
14:06 pdurbin
thanks
14:10 stefankasberger4
@pdurbin: yes. what's u?
14:11 pdurbin
stefankasberger4: have you heard of swagger?
14:15 poikilotherm
pdurbin: just yesterday I saw that OpenAPI is pretty successfull. And Hackolade announced a GUI for designing/drafting with Swagger/OpenAPI
14:15 pdurbin
oh? do you have a link?
14:15 poikilotherm
Gimme a sec
14:16 poikilotherm
https://github.com/hackolade/Swagger
14:16 poikilotherm
This seems to be the project behind it
14:16 pdurbin
The reason I ask is because on Monday I met with some developers here and one of them said that if we provide a swagger/openapi file that it would be easier for him to create a dataverse-client-javascript for us.
14:17 poikilotherm
Totally agree
14:17 stefankasberger4
yes. worked with it. and use it with an Django microservice (there is a great djanto extension for swagger / openAPI support).
14:18 poikilotherm
https://hackolade.com/help/SwaggerAPI.html
14:18 stefankasberger4
https://www.openapis.org/
14:18 poikilotherm
I do own a license for Hackolade IIRC
14:18 poikilotherm
Unsure if it is still valid
14:19 pdurbin
So here's my request. Can someone create an issue saying that Dataverse should provide this Swagger/OpenAPI file? stefankasberger4 and poikilotherm can arm wrestle over the honor of creating that issue. :)
14:19 poikilotherm
I'll leave the honors to stefankasberger4
14:19 poikilotherm
I am not currently using this
14:19 pdurbin
stefankasberger4: do you mind creating an issue for this?
14:20 stefankasberger4
https://swagger.io/docs/specification/about/
14:35 pdurbin
stefankasberger4: or I can ask someone else to do it. Thoughts?
14:36 pdurbin
donsizemore: that reminds me. We can build doc type stuff with the new jenkins if you want. javadoc, schemaspy, mvn site, etc.
14:36 pdurbin
not to distract you with the million and first thing :)
14:38 stefankasberger4
will do.
14:38 * poikilotherm
*thumbs up*
14:40 pdurbin
thanks!!
14:40 poikilotherm
This might be usefull for the CESSDA/DataverseEU people, too. That "Dataverse App Store" will most certainly have apps that need an API and Swagger could be of great help
14:41 poikilotherm
https://github.com/IQSS/dataverse/issues/5688
14:41 pdurbin
It can't hurt. And we already have a commit in a random unmerged branch for how to add the javadoc annotations to the JAX-RS endpoints.
14:42 pdurbin
donsizemore: and we can have jenkins build the swagger docs :)
14:42 stefankasberger4
https://github.com/IQSS/dataverse/issues/5794
14:42 poikilotherm
https://tenor.com/search/muahaha-gifs
14:43 pdurbin
stefankasberger4: thanks!
14:43 stefankasberger4
you are welcome!
14:49 pdurbin
I found the Swagger scratch work. I'll push it to a branch under the IQSS repo.
15:06 pdurbin
ok, pushed and commented
15:06 pdurbin
thanks again
17:00 donsizemore joined #dataverse
17:02 donsizemore
@pdurbin i'm happy to make jenkins build whatever you'd like for jenkins to build
17:03 donsizemore
also, i'm on a BagIt call with Jim Myers and Sebastian
17:10 pdurbin
oh? :)
17:10 pdurbin
tell 'em I said hey
18:57 donsizemore joined #dataverse
19:52 donsizemore
@pdurbin c.f. https://github.com/IQSS/dataverse-ansible/commit/d1fa9d1120cf7ffe5536da9f86fa4892781d1cef
19:52 donsizemore
want to test again tomorrow morning, then will merge
19:53 donsizemore
have a great afternoon!
19:53 pdurbin
yay group vars! thanks!
21:28 pameyer joined #dataverse
21:29 pameyer
pdurbin - I know that https://github.com/IQSS/dataverse/issues/1936 is closed, but if you've got credentials there might be a way to get the config xml out
21:29 pameyer
wget https://build.hmdc.harvard.edu:8443/jnlpJars/jenkins-cli.jar
21:30 pameyer
java -jar jenkins-cli.jar -s https://build.hmdc.harvard.edu:8443/ get-job phoenix.dataverse.org-apitest-develop
21:30 pameyer
I get `anonymous is missing the Job/ExtendedRead permission` ; but you can auth to jenkins - so you _should_ (or maybe might) get xml out
22:11 jri joined #dataverse
22:42 pdurbin
huh, interesting, thanks, Pete!
