Time
S
Nick
Message
09:02 juancorr joined #dataverse
11:07 jri joined #dataverse
13:22 donsizemore joined #dataverse
13:22 donsizemore
@pdurbin knock knock?
13:28 pdurbin
what's the good word donsizemore
13:29 pdurbin
oooo, I see "true"|lower
13:29 donsizemore
@pdurbin how wed are you to the ec2-create script being a standalone with only command switches? in this case it's rickety and unwieldly
13:29 donsizemore
@pdurbin how would you feel about keeping common (repo, branch) flags, then specifying an optional group_vars file for the ec2-script to call with ansible-playbook?
13:30 donsizemore
@pdurbin particularly now that the DOI parameters are less of a consideration
13:30 pdurbin
I think I'm find with whatever but would the new group_vars file be different than the existing .dataverse ec2 env file?
13:31 donsizemore
it would be a lot cleaner than a mix of flags and a set .env file — just send ansible an arbitrary group_vars file if you don't want it to use the repo's defaults
13:31 donsizemore
@pdurbin just grab a copy and edit it prior to running ec2-create, which would buck your preference for a single script with flags
13:32 pdurbin
So it would be a different file, a new file. So we'd have three files in play? The script itself, the existing env file, and a new group_vars file?
13:33 donsizemore
i'd ditch the env file
13:33 donsizemore
in hindsight i should've gone with an external group_vars file anyway
13:33 pdurbin
Well, now we're using it in anger. :)
13:34 donsizemore
where's anger?
13:34 pdurbin
What did the Hulk say? I'm always angry?
13:36 donsizemore
i was more of an archie / bloom county comics kid
13:37 pdurbin
This is good timing, actually. Just yesterday we talked about how the design team isn't benefiting from the spin up script and we'd like to them to. Can we dream about a button they would click? Maybe in Jenkins? I don't know.
13:37 donsizemore
anyway, branch and repo have served us well, but even in the best case additional vars got messy quickly
13:37 donsizemore
jenkins could absolutely do this
13:38 pdurbin
cool
13:38 pdurbin
So wait, -b and -r go away too?
13:39 donsizemore
those are pretty basic and common (and ansible doesn't try to (sometimes) turn them into booleans)
13:40 pdurbin
Ok, so maybe they can stay.
13:40 donsizemore
but IMHO the messier stuff which would change less often could/should go in a file rather than the CLI , and group_vars is already mostly complete there.
13:41 donsizemore
anyway, that's my proposal for a cleaner ec2/ansible combo
13:42 pdurbin
Sounds great. Want me to create an issue? And you make the pull request? :)
13:42 donsizemore
i'll do all that. just wanted permission to avoid asking for forgiveness
13:43 pdurbin
Great! Thank you! And the button should be out of scope but let's keep talking about it. Pete might have some ideas here. He spins up Jenkins on his laptop.
13:44 donsizemore
i do like buttons. we have a dataverse-related project here that builds from a github webhook, and on success deploys the warfile to a test VM . we could easily do something similar with ec2
13:45 donsizemore
so it'd have to be an oversized bookbag button, large enough to hold the desired options
13:46 pdurbin
NDS Labs Workbench has a button. Are you aware of it?
13:47 pdurbin
Lots of buttons.
13:47 donsizemore
is it public?
13:48 pdurbin
Yes. Please see https://github.com/IQSS/dataverse/issues/4152
13:50 donsizemore
this begs the question. where would the ec2 button live?
13:51 pdurbin
I was thinking Jenkins but I haven't thought about it deeply.
13:52 pdurbin
Some little PHP app behind HTTP auth? I don
13:52 pdurbin
don't know.
13:52 pdurbin
Python destop app?
13:52 donsizemore
well, let me start with tidying up the group_vars stuff. you saw i made another poke at solr yesterday. any particular snafu you noticed in your usage?
13:53 pdurbin
I have a suspicion that one can't create dataverses and datasets due to Solr stuff. Not sure.
13:54 donsizemore
well, now that i knocked monday out of the way i can do my own testing
13:54 pdurbin
It would be great to start using the EC2 spin up stuff with some API tests. Then we'd know if there are any failures.
13:56 pdurbin
donsizemore: I just had a thought. If the EC2 script is becoming more Ansible-specific, should it live in the dataverse-ansible repo instead?
13:57 donsizemore
either way. that would cut out a bit of branch coordination. again i'd love for them all to go into a "toolbox" somewhere
14:00 pdurbin
Want me to create a dataverse-toolbox repo?
14:01 donsizemore
now that we've got ansible, docker, k8, and ec2 bits flying around i could see that being handy
14:02 donsizemore
it would keep your core codebase cleaner as well
14:04 pdurbin
The tools could move faster that way. Our process for the main code base is heavy.
14:05 pdurbin
I think for now we should keep the status quo or consider moving the script to dataverse-ansible. And think some more about what would go in the toolbox repo. I could create a Google doc to captures ideas for a new repo if you want.
14:47 donsizemore joined #dataverse
15:03 donsizemore
@pdurbin i'm on board with all of the above =)
15:16 pdurbin
donsizemore: cool, I just started a "Dataverse toolbox" doc. Please request edit access: https://docs.google.com/document/d/1iuxX0suRdoKCd3k6YmXNXfhf-kT7EbuQEzi9NYts6M0/edit?usp=sharing
15:16 pdurbin
Did you catch Jim mentioning Chef yesterday?
15:21 donsizemore
i've used puppet but not chef. the research computing guys here love salt
15:23 pdurbin
I've used Puppet the most but that was years ago. A pinch of Salt. Happy to learn some Ansible.
15:46 pameyer joined #dataverse
16:14 pdurbin
The community call is in about 45 minutes: https://dataverse.org/community-calls
16:15 pdurbin
andrewSC bjonnh bricas donsizemore jri juancorr pameyer pmauduit xarthisius yoh ^^
18:22 jri joined #dataverse
19:54 Jim__ joined #dataverse
19:54 pdurbin
Jim__: thanks for calling in today
19:55 Jim__
sure - fun as usual...
19:55 Jim__
Did you see my lost file issue yet? Could be lots of fun if people were publishing...
19:58 pdurbin
Nope but I just found it: https://github.com/IQSS/dataverse/issues/5469
20:24 donsizemore
@pdurbin i haven't forgotten the ec2 script, just stuff(tm) came up today. it's in progress
20:47 pdurbin
donsizemore: speak of the devil. We were just talking about that script. Any thoughts on how to get test data into an installation? A few users, datasets, files, etc.
20:55 donsizemore
@pdurbin just submitted a PR for your review. should allow full group_var support from a local copy of the file
20:55 donsizemore
@pdurbin you had trees and insects or something in your test data stuff on phoenix(?)
20:55 donsizemore
@pdurbin i can include that as a switch next if you'd like
20:57 pdurbin
Yeah, birds and trees. Created with "create-all-and-test" at https://github.com/IQSS/dataverse/blob/v4.10.1/scripts/deploy/phoenix.dataverse.org/post#L14 . There's still there. It runs on every build.
20:59 donsizemore
IIRC i stopped at publishing the root dataverse (which should probably be a group_var)
20:59 donsizemore
but i'm happy to add in some optional test JSON and switches to create things if need be?
21:00 pdurbin
That would be awesome. People don't want birds and trees but maybe you could use my scripts as a starting point? Up to you.
21:10 pdurbin
donsizemore: I'm on your branch trying things out. Thanks!
21:11 donsizemore
@pdurbin "it worked on my laptop"
21:14 pdurbin
Heh. I'm rewriting the "Configure ~/.dataverse/ec2.env" stuff at http://guides.dataverse.org/en/4.10.1/developers/deployment.html
21:26 donsizemore
oh. documentation. that.
21:28 pdurbin
donsizemore: I just pushed my doc change: https://github.com/IQSS/dataverse/commit/3bdf6c9c5
21:29 pdurbin
Please feel free to edit further!
21:34 pdurbin
donsizemore: huh, I passed `-b develop` but it's skipping the building of the war file. Is that still the right way to make it build an arbitrary branch?
21:34 donsizemore
poop. lemme fix that.
21:34 pdurbin
thanks!
21:35 donsizemore
yeah, i pulled too much out
21:58 donsizemore
@pdurbin http://ec2-3-84-19-220.compute-1.amazonaws.com:8080
22:53 jri joined #dataverse
