Time
S
Nick
Message
09:45 poikilotherm joined #dataverse
12:59 donsizemore joined #dataverse
13:00 pdurbin
donsizemore: oh hey, I was just mentioning you here in a comment: https://github.com/IQSS/dataverse/issues/4990#issuecomment-423522400
13:03 donsizemore
any chance we could just try an ansible update? i also like your suggested_repo additions
13:06 pdurbin
So I would enable EPEL before installing Ansible?
13:14 donsizemore
yes, it's part of the epel repo
13:21 * pdurbin
tries it
13:30 pdurbin
donsizemore: how does Ansible 2.6.4-1.el7 sound?
13:31 donsizemore
excellent =) (for some reason i see 2.6.3 in ibiblio's epel mirror)
13:31 donsizemore
no, that's just the metadata. it wants to install 2.6.4
13:37 pdurbin
donsizemore: the problem is that --extra-vars doesn't work
13:38 donsizemore
is it ignoring the -e you're passing, or throwing an error?
13:38 pdurbin
The 4.9.2 release is being deployed rather than the branch I specified.
13:38 donsizemore
can you send me the output from the run?
13:38 pdurbin
yep, one sec
13:39 pdurbin
I can even let you ssh into the ec2 instance if you want.
13:44 pdurbin
donsizemore: I just emailed you the output and the private key. No bitcoin mining, please. :)
13:45 donsizemore
muahahahahaaaaa
13:45 pdurbin
heh
13:47 donsizemore
yeah, it ignored your --extra-vars (and it skipped two solr tasks it shouldn't have skipped)
13:50 donsizemore
@pdurbin a) are you running the version of the script at https://github.com/IQSS/dataverse/blob/4990-ec2-ansible-scripting/scripts/installer/ec2-create-instance.sh and are you passing the extra_vars as 'dataverse.git.branch' and 'dataverse.git.repo'? (just double-checking)
13:50 donsizemore
oh wait, i should SSH in
13:51 pdurbin
donsizemore: I'm not because it didn't work.
13:51 pdurbin
I tried many things including passing in JSON .
13:54 donsizemore
okay. will you do me a favor, and re-run the script (or i can if you like) and ask ec2-create-instance.sh to echo the ansible-playbook command before running it? the simplest answer is a variable mismatch
13:55 pdurbin
sure, echoing
13:56 pdurbin
This takes a while.
13:57 pdurbin
ansible-playbook -i dataverse/inventory dataverse/dataverse.pb --connection=local --extra-vars branch=4610-upload-dual-mode repo=https://github.com/IQSS/dataverse.git
13:58 pdurbin
donsizemore: ^^
14:21 pameyer joined #dataverse
14:23 pameyer
pdurbin: does using the `--check` option on the ansible-playbook call produce anything useful?
14:24 pdurbin
pameyer: I could try that
14:27 pdurbin
I'd love for you or Don to try the script. :)
14:28 pdurbin
So far it seems like adding --check doesn't have an effect. It's just running the playbook like normal.
14:31 pdurbin
weird, adding --check let to this error: TASK [dataverse : create glassfish postgres database] ************************** fatal: [localhost]: FAILED! => {"changed": false, "msg": "the python psycopg2 module is required"}
14:31 pdurbin
led*
14:31 pameyer
ah - not too shocking.
14:32 pameyer
check shouldn't do anything; which means it hasn't installed the dependency it needs to check downstream stuff
14:32 pameyer
oddness
14:32 pameyer
but doesn't fail with a syntax/parsing error
14:35 pdurbin
pameyer: what's your reading of https://docs.ansible.com/ansible/2.6/user_guide/playbooks_variables.html#using-variables-about-jinja2 in terms of if I'm on the right track adding variables in curly braces to the ansible side such as '{{ branch }}'
14:36 pdurbin
Don doesn't seem to like the curly braces but it seems to be the only way to get --extra-vars to work.
14:36 donsizemore joined #dataverse
14:37 pameyer
pdurbin: that I don't think you're trying to *use* the variables, you're trying to assign non-default values to them
14:37 donsizemore
@pdurbin it's not that i don't like the curly braces, it's that if anyone runs ansible-playbook without explicitly assigning those variables the role breaks
14:38 pdurbin
donsizemore: yeah, like in Travis, which I broke
14:38 donsizemore
@pdurbin we could just add those two --extra-var arguments to the usage instructions to call it good, but if ansible would respect variable precedence we'd be done
14:38 pameyer
double curly braces is the normal jinja2 templating (from the rendering context); but what you want to do is change the variable in the context prior to rendering
14:39 pdurbin
donsizemore: how come you keep giving me dotted examples like "dataverse.git.branch" but there are no dotted examples in the Ansible docs?
14:41 pameyer
I'd guess because that's how you'd access nested dictionaries from a jinja2 template
14:41 pdurbin
Yeah but have either of you tested it? It's not documented and it doesn't work.
14:42 pdurbin
In my mind adding a curly brace variable like {{branch}} is the hook I need to use --extra-vars. Otherwise, it just doesn't work.
14:42 pameyer
which suggests I'm guessing wrong
14:42 pameyer
haven't tested it in that role; but that's how it works in jinja2
14:42 pameyer
and I have used it there
14:43 pdurbin
ok, do either of you want to give the script a try?
14:45 donsizemore
@pdurbin that's why i was asking about what you were passing to ansible-playbook: i'd pass dataverse.git.branch and dataverse.git.repo. i haven't tested it because i hadn't yet because i went home yesterday evening. okay for me to run yours as-is, or should I request permission to spend some $$$ on AWS (another department on campus is paying our bill)
14:45 donsizemore
@pdurbin i was really asking about the full output of the run, as the script only produced stdout and for debugging seeing the exact ansible-playbook command would be very helpful.
14:45 pdurbin
donsizemore: let me see if I can create an account on our AWS stuff
14:47 donsizemore
@pdurbin that would be awesome, thank you!
14:47 pdurbin
donsizemore: I see an "Add User" button. What do you want your username to be?
14:51 donsizemore
dls or donsizemore would be wonderful
14:51 pdurbin
I'll go for donsizemore so people here know who it is.
14:54 pdurbin
donsizemore: ok, please check your email
14:56 donsizemore
many thanks, cloning the 4990 branch now =)
14:56 pdurbin
perfect. thanks!
14:57 pdurbin
donsizemore: I got it working with non-IQSS repos, by the way. Thanks for adding the "repo" config
14:57 donsizemore
no, thank you. this stuff is fun but without an instance i could only guess
14:57 pdurbin
It's good for me to go through the process of creating an AWS account because we'll need to create one for Tania, Derek, and others who have expressed interest in spinning up arbitrary branches.
14:58 pdurbin
donsizemore: now that you're in there I'll be a little more careful with the ec2-destroy-all.sh script, which I just ran. :)
15:00 donsizemore
@pdurbin won't hurt me none
15:01 pdurbin
well, I don't want to destroy an EC2 instance you're in the process of spinning up. I'll check using ec2-list-all.sh first.
15:06 donsizemore
@pdurbin blindly firing away, as I do...
15:07 pdurbin
only way to fly
15:07 pdurbin
donsizemore: I just want you to feel my pain :)
15:13 donsizemore
@pdurbin i'm being obstinate/methodical and running it as-is first
15:13 pdurbin
perfect. should work!
15:14 pdurbin
I can see your instance with ec2-list-all.sh
15:24 donsizemore
@pdurbin while we wait, may i ask a pesky question?
15:35 donsizemore
@pdurbin so mine spun up 5009-svg-footer at http://ec2-52-91-163-246.compute-1.amazonaws.com:8080 as evidenced by http://ec2-52-91-163-246.compute-1.amazonaws.com:8080/resources/images/dataverse_project_logo.svg (i think?)
15:35 donsizemore
@pdurbin so mine spun up 5009-svg-footer at http://ec2-52-91-163-246.compute-1.amazonaws.com:8080 as evidenced by http://ec2-52-91-163-246.compute-1.amazonaws.com:8080/resources/images/dataverse_project_logo.svg
15:36 donsizemore
@pdurbin so mine spun up 5009-svg-footer at http://ec2-52-91-163-246.compute-1.amazonaws.com:8080
15:36 donsizemore94 joined #dataverse
15:36 donsizemore94
sorry 4 spamming the channel, my message wasn't returning to me...
15:37 pdurbin
sorry, was at standup
15:38 pdurbin
yep, the SVG branch. worked
15:38 pdurbin
I'm heading to lunch in about 10 minutes but hopefully I can answer your pesky question. :)
15:59 donsizemore94
@pdurbin trying again with 5027-dashboard-admin-guide
16:32 dataverse-user joined #dataverse
17:12 pdurbin
cool
17:14 donsizemore94
@pdurbin woo woo i deployed 5027-dashboard-admin-guide to http://ec2-18-212-19-147.compute-1.amazonaws.com:8080
17:15 donsizemore94
so... this was without my accepting your PR to dataverse-ansible...?
17:19 pdurbin
donsizemore94: if you look at the script you'll see that I'm switching to the "extra-vars-travis" branch I pushed to the dataverse-ansible repo. I'm depending on that {{branch}} and {{repo}} I added.
17:25 donsizemore94
@pdurbin ah. saw that line and it didn't click. (full disclosure: we had a pizza party for our new interim director)
17:26 pdurbin
heh
17:26 pdurbin
you should see some FIXMEs in the script mentioning the pull request
17:26 pdurbin
So it all works great. :)
17:26 pdurbin
You just need to merge that pull request. :)
17:36 donsizemore94
@pdurbin i'm trying some tweaks to the ansible-playbook line without the travis branch
17:56 pdurbin
donsizemore94: great! Thank!
17:56 pdurbin
s
18:45 donsizemore94
@pdurbin (and i should have it working this afternoon)
18:46 pdurbin
awesome
19:21 dataverse-user joined #dataverse
19:21 dataverse-user
hi
19:22 pameyer
hi dataverse-user
19:22 dataverse-user
hello, what's up
19:27 pameyer
well, do you have an opinion on jinja2 rendering vs context and ansible extra args?
19:29 dataverse-user
no,
19:29 dataverse-user
no idea what you're talking about
19:30 pameyer
fair enough :) it's a bit into the guts of dataverse
19:30 pameyer
but that's what was up this morning, at least
19:30 dataverse-user
whats the dataverse
19:30 dataverse-user
a bunch of data scientists here or something
19:32 pameyer
https://dataverse.org
19:34 pdurbin
pameyer is an actual scientist :)
19:34 dataverse-user
typo in on the `about the project` page
19:34 dataverse-user
and my sentence
19:38 pdurbin
no worries
19:42 dataverse-user
interesting
19:52 donsizemore94
@pdurbin meow? http://ec2-174-129-180-167.compute-1.amazonaws.com:8080
19:52 pdurbin
donsizemore94: 5009-svg-footer. good. You fixed it?
19:54 donsizemore94
@pdurbin you and pete were right. extra vars apparently can't be nested, still. so i renamed those two in dataverse-ansible and in the ec2-create script
19:54 donsizemore94
@pdurbin also stuck a yum -y update ansible just before the ansible-playbook call (for some reason, having it earlier in the EOF block didn't update it in the running instance
19:54 donsizemore94
@pdurbin anyway, a git diff is in your inbox. changes to two lines and you should be good?
19:56 pdurbin
it sure is. Christmas came early!
19:56 donsizemore94
@pdurbin i would've been done sooner but life happens, even at work
19:57 pdurbin
It's much appreciated. Lemme try it out. I see master on dataverse-ansible has already been updated.
19:58 donsizemore94
@pdurbin that's essentially what i changed to fix ec2-create =)
19:58 pdurbin
donsizemore94: do you still want the newer version of Ansible? The one from EPEL?
19:58 donsizemore94
you should have it now. moving it down a few lines in the script made a difference for some reason
19:59 donsizemore94
argh, just checked again. no you don't, but you don't seem to need it anyway
19:59 pdurbin
Ok. I guess the question is if you care either way.
20:00 donsizemore94
i only care when things are broken. fix it fix it and all that
20:02 pdurbin
Your playbook installs EPEL anyway, right? So if someone gets on the machine and runs `yum update`, Ansible will be updated from the version in base Centos to the version in EPEL, right? If so, I'm thinking we should just use the version from EPEL.
20:07 donsizemore94
@pdurbin i pulled it from the playbook; the update call worked in vagrant but not quote-unquote bare metal
20:08 donsizemore94
@pdurbin and i had included it in part as a matter of policy but mostly to work around earlier version discrepancies that they've since ironed out
20:08 pdurbin
ok, but you do enable EPEL?
20:08 donsizemore94
twice IIRC. in anticipation of splitting out the roles across machines
20:08 pdurbin
heh. ok
20:10 pdurbin
waiting for this: TASK [dataverse : build warfile. tail /tmp/dataverse/mvn.out for gory details.] ***
20:10 pdurbin
any thoughts on how to speed that up?
20:13 donsizemore94
use an AWS image with more cores ;)
20:14 donsizemore94
or honestly you spend most of that step downloading maven dependencies. a zipped local repo or something to avoid the download step would greatly speed warfile compilation
20:17 pdurbin
hmm, but we'd have to update the zipped local repo from time to time
20:18 donsizemore94
@pdurbin correct. but that could be periodically automated or something. anyway, the majority of that step is dependency download
20:19 pdurbin
yeah
20:21 pdurbin
donsizemore94: it works! Thank you!!
20:22 donsizemore94
excellent.
20:24 donsizemore94
i'm sneaking away for cider, hope you have a great weekend!
20:24 pdurbin
you too! enjoy!
22:16 pameyer left #dataverse
