Time
S
Nick
Message
02:18 garnett joined #dataverse
04:56 garnett joined #dataverse
06:51 jri joined #dataverse
12:24 donsizemore joined #dataverse
13:32 pameyer joined #dataverse
14:32 donsizemore joined #dataverse
15:22 garnett joined #dataverse
15:29 garnett joined #dataverse
16:10 axfelix joined #dataverse
16:48 nicholas_ joined #dataverse
17:01 axfelix joined #dataverse
17:20 axfelix joined #dataverse
17:57 pdurbin
telnoratti: I just created an AWS account but now I'm waiting to be added to consolidated billing.
18:18 axfelix joined #dataverse
18:34 telnoratti
Cool, I just got the authenticated relay set up. We're using SES, but I see no reason why you couldn't use some external relay
18:35 telnoratti
still trying to figure out the SSL cert problem though
18:45 axfelix joined #dataverse
18:53 pdurbin
telnoratti: ok, billing is all set. Now to look at what you wrote at http://irclog.iq.harvard.edu/dataverse/2016-09-30 :)
18:54 telnoratti
I'll be present on irc until a little before 17 today, maybe I can get joel in here too
18:56 joelmarkanderson joined #dataverse
18:57 pdurbin
telnoratti: I'm not sure what to put in "Glassfish discovery URL
18:57 telnoratti
read the help text on it, basically go to https://discovery.etcd.io/new?size=3 which generates a discovery url for you
18:58 telnoratti
and copy and paste that into the box, it's just the initial mechanism coreos uses to establish a cluster
18:58 pdurbin
It says "An unique etcd cluster discovery URL . Grab a new token from https://discovery.etcd.io/new?size= <your cluster size>". Oh, maybe you wrote that. Maybe it came from the JSON .
18:59 telnoratti
yeah, that first page is all inputs mentioned in dataverse.json
18:59 pdurbin
I guess these tags are optional.
19:00 pdurbin
ok. CREATE_IN_PROGRESS
19:00 pdurbin
joelmarkanderson: oh hi!
19:01 joelmarkanderson
hi philip
19:01 pdurbin
ROLLBACK_COMPLETE
19:02 telnoratti
oh >.<
19:02 telnoratti
What was the error?
19:02 telnoratti
you should be able to find that by looking at the events and finding the first create that failed
19:03 pdurbin
"Parameter validation failed: parameter value for parameter name KeyPair does not exist. Rollback requested by user."
19:03 telnoratti
Oh, you'll need to create a keypair in EC2
19:04 telnoratti
so under the EC2 service and under network & security on the left you'll find the spot you can upload a keypair
19:04 telnoratti
once you have some uploaded, that field will be a drop down menu
19:06 pdurbin
ah, under NETWORK & SECURITY there's "Key Pairs"
19:07 joelmarkanderson
that's what you want; then i download and ssh -i keypair.file core
19:08 pdurbin
DELETE_IN_PROGRESS
19:08 pdurbin
whoops, now it's gone
19:09 joelmarkanderson
don't get too attached to any one stack
19:09 pdurbin
ok
19:09 pdurbin
but it was my first stack
19:12 pdurbin
ah, the keypair thing is at the bottom of the template thing
19:13 telnoratti
yeah it sneaks in there, one of the items on our todo list is to polish that form
19:13 joelmarkanderson
this helps us highlight which parameters are truly mandatory
19:13 pdurbin
can you make the keypair mandatory?
19:13 telnoratti
I'm not sure
19:16 telnoratti
looks like we can specify minlength is 1
19:16 telnoratti
might be the most efficient mechanism, it should already be validating any entered values
19:18 pdurbin
better than nothing I guess
19:19 telnoratti
yeah we're taking notes on what needs to be documented
19:20 axfelix joined #dataverse
19:22 pdurbin
I forgot to change whatever the default rollback setting is but I've heard this is important.
19:23 pdurbin
CREATE_COMPLETE
19:23 pdurbin
\o/
19:24 pdurbin
but how do I get to it in a browser?
19:27 pdurbin
`ssh core
19:28 joelmarkanderson
once the load balancer has a glassfish node behind it, paste its DNS into your browser
19:28 telnoratti
if you click on the stack, then expand the resources section you can then click on the GlassfishLB
19:28 telnoratti
that'll pull it up in the console so you can find out what the DNS is
19:29 telnoratti
this should probably be an output
19:32 pdurbin
`curl -i foobarbaz-Glassfis-1M9B1YZL8P58I-1600615884.us-west-2.elb.amazonaws.com` shows "HTTP /1.1 503 Service Unavailable: Back-end server is at capacity"
19:33 telnoratti
It takes a little time to do the install
19:33 pdurbin
oh
19:33 telnoratti
you might be waiting on that
19:33 joelmarkanderson
check for your node availability
19:33 telnoratti
you can ssh into one of the nodes and do `systemctl status glassfish`
19:34 joelmarkanderson
(aws console) under ec2->load balancer, look for status: x of y instances in service
19:35 pdurbin
"0 of 3 instances in service"
19:35 pdurbin
all three are "OutOfService"
19:37 pdurbin
telnoratti: should I run `systemctl status glassfish` as "core" or "root" (or some other user?)
19:37 joelmarkanderson
login as core, sudo as root
19:38 joelmarkanderson
if they take too long to join your balancer, start digging logs
19:38 telnoratti
`journalctl -u startup` `journalctl -u glassfish` and `etcdctl get /dvninit` will be your friends
19:41 pdurbin
here's what I see from systemctl and journalctl and etcdctl so far: https://paste.fedoraproject.org/444363/
19:41 telnoratti
startup is responsible for running the install script. It runs /home/core/startup.sh which uses atomic operations on /dnvinit to syncronize. Only one of the nodes will be running the install script, so you may have to ssh into each machine to find the one with unique `journalctl -u startup`
19:41 joelmarkanderson
i think you're building a bad commit
19:42 telnoratti
oh that doesn't look right at all
19:43 telnoratti
what does `journalctl -u docker` say?
19:43 pdurbin
this is the commit I'm on: https://github.com/IQSS/dataverse-aws/commit/32fa7a8
19:45 joelmarkanderson
i'm building the same one right now, let me look
19:45 pdurbin
`journalctl -u docker` isn't looking so good: https://paste.fedoraproject.org/444368/
19:48 joelmarkanderson
we're us-east-1; where are you?
19:48 telnoratti
oh yup, we apparently hardcoded us-east-1 in a few places
19:48 pdurbin
Boston (Cambridge)
19:49 joelmarkanderson
we'll parameterize that too
19:49 telnoratti
looks like you're in us-west-2
19:49 pdurbin
oh
19:50 telnoratti
you should be able to change your region at the top right of the console
19:51 pdurbin
if I click "Oregon" in the top right there's a dropdown. Looks like I'm in US West (Oregon).
19:52 telnoratti
yeah, swap that to N. Virginia until we get the region thing fixed
19:53 telnoratti
it doesn't really matter which one your in as long as it's one of the big ones
19:53 joelmarkanderson
i should push a solution in a moment
19:54 telnoratti
some of the smaller/newer regions don't have all features
19:55 pdurbin
I guess I should delete this running stack.
19:56 telnoratti
it will take a little to delete as well
19:57 pdurbin
DELETE_IN_PROGRESS
19:59 joelmarkanderson
pushed new commit
19:59 joelmarkanderson
should fix install process and fetch your current regio
19:59 joelmarkanderson
n
19:59 joelmarkanderson
we ran into a bottleneck of the default VPC limit of 5
20:00 joelmarkanderson
keep in mind when building multiple stacks
20:00 joelmarkanderson
plus other cloud development efforts we juggle
20:00 pdurbin
joelmarkanderson: thanks, I just pulled and will try again with 4cf2520 after the delete is done, I guess.
20:04 joelmarkanderson
perfect
20:05 pdurbin
joelmarkanderson: you want me to keep in on Oregon?
20:05 pdurbin
it*
20:06 joelmarkanderson
i feel like something was restricted to us-east-1, but that may not be current now
20:06 joelmarkanderson
all services we need might be global now
20:06 telnoratti
pdurbin: no need to wait, you can create a new stack while the old one is being deleted
20:06 telnoratti
we usually have several at any time
20:10 pdurbin
joelmarkanderson: I'll try switching it from Oregon to N. Virginia just in case
20:10 telnoratti
yeah, we can test that ourselves
20:10 joelmarkanderson
it's worth knowing if oregon will work later
20:11 pdurbin
"Template validation error: Template format error: Unresolved resource dependencies [AdminEmail] in the Resources block of the template"
20:11 telnoratti
yeah I bumped into that, I'll push my fix shortly
20:11 pdurbin
ok
20:14 telnoratti
newest commit should work
20:15 * pdurbin
pulls 208cfc9
20:16 pdurbin
the keypair field is a little funny now
20:16 pdurbin
before it was a dropdown
20:17 joelmarkanderson
side effect of "MinLength": "1"?
20:17 telnoratti
oh keypairs might be restricted to region
20:17 joelmarkanderson
of course
20:17 telnoratti
aws regions are pretty close to entirely independant
20:18 pdurbin
so should I use N. Virginia or Oregon?
20:18 joelmarkanderson
either one - just migrate your keypair
20:18 telnoratti
I would just use N. Virginia until we're sure other regions work
20:19 pdurbin
KeyPair won't let me enter a value anymore.
20:20 telnoratti
did you add a keypair while in the us-east-1 region?
20:20 telnoratti
I swear I tested the commit
20:21 pdurbin
I'll go add it.
20:21 telnoratti
yeah aws regions share basically nothing
20:21 telnoratti
it's kind of weird
20:24 pdurbin
I'm changing rollback on failure from yes to no.
20:34 pdurbin
ok, I gotta run but fingers crossed that http://virginia-Glassfish-1MKGKZFE19LRZ-1902100397.us-east-1.elb.amazonaws.com comes up ok
20:34 pdurbin
or https://virginia-Glassfish-1MKGKZFE19LRZ-1902100397.us-east-1.elb.amazonaws.com (https)? I guess we'll see! :)
20:34 pdurbin
telnoratti joelmarkanderson: thanks for all your help!
20:35 pdurbin
once we get it working we can upgrade it to Dataverse 4.5.1: https://github.com/IQSS/dataverse/releases/tag/v4.5.1 :)
20:35 pdurbin
just deployed to https://dataverse.harvard.edu ;)
20:42 telnoratti
should be easy enough, assuming the install script didn't change too much
20:42 pdurbin
telnoratti: I don't think it changed. We haven't gotten around to your pull request.
20:43 pdurbin
nice, https://github.com/IQSS/dataverse-client-java exists now :)
20:43 pdurbin
ok, really going
21:13 axfelix joined #dataverse
21:23 axfelix joined #dataverse
22:59 pdurbin
hmm, "0 of 3 instances in service" again
23:08 axfelix joined #dataverse
23:21 axfelix joined #dataverse
