Time
S
Nick
Message
03:25 jri joined #dataverse
08:01 jri joined #dataverse
08:03 jri_ joined #dataverse
12:45 pdurbin joined #dataverse
13:06 donsizemore joined #dataverse
13:11 donsizemore joined #dataverse
13:51 donsizemore
@pdurbin knock knock?
13:51 pdurbin
who's there?
13:51 donsizemore
@pdurbin frozen
13:51 donsizemore
(and i'm south of the mason-dixon)
13:51 donsizemore
quick Q, if you're at work today
13:52 pdurbin
-5F air temp at Sunday River yesterday. -25F with wind chill. Took the kids skiing anyway. Good times. Fresh powder.
13:53 donsizemore
the weather channel put up a webcam from holy cross. very nice to look at indoors with the furnace running
13:54 pdurbin
Heh. I'm working from home with a sick kid (too sick to ski yesterday). You said you have a question?
13:55 donsizemore
so, i was going to tell dataverse-ansible to test for a directory containing sample data. if the directory doesn't exist, it pulls your create-* stuff from github. if the directory does exist, attempt to import in order, assuming a 00_ 01_ naming convention or some such.
13:55 donsizemore
overkill for sample data? would allow fine-tuning for those who desired it
13:56 pdurbin
How would the sample data get into place?
13:58 donsizemore
if there isn't any, we pull yours from github. if there are extant files in the sample data directory, attempt to use them
13:59 pdurbin
Sure, I get that and it sounds fine but do we wget the sample data directory from somewhere? Do we check the sample data into a git repo?
14:01 donsizemore
@pdurbin i was just going to make it a group_var and expect cardinal filenames so dataverses would get created before datasets. would allow any amount of custom sample data if anyone wants to write their own JSON
14:01 donsizemore
@pdurbin but yes, if the directory doesn't exist create it and 'curl -O' your existing sample data from github
14:02 pdurbin
Sounds fine but I guess what I
14:03 pdurbin
I'm asking is... if I want to write some custom JSON , how do I put it into place? Is this a job for the ec2-create script?
14:04 donsizemore
@pdurbin that would need to pre-exist (as would the group_vars file for someone who would want to customize it)
14:06 pdurbin
Ok, but you wouldn't object to me further hacking on the ec2-create script when you're done. I could add a feature to download some sample data from somewhere before the Ansible run.
14:06 donsizemore
i could make that a group_var is well. (zip file maybe?)
14:08 pdurbin
Yeah, I was thinking zip file or tarball. I'm more of a tar person, to be honest.
14:08 donsizemore
spoken like a true colonial ;)
14:08 pdurbin
heh
14:09 pdurbin
How do we get this to work from Vagrant?
14:09 pdurbin
Or the phoenix server?
14:09 pdurbin
Is this an Ansible-only feature?
14:10 pdurbin
I'm not criticizing. I'm just thinking out loud.
14:10 donsizemore
um, i was going to put it in ansible since vagrant calls it. in that way, it's not "ec2-only" and would mostly depend on group_vars
14:11 pdurbin
Ah, well, I was thinking about the *other* Vagrantfile. The one in the "dataverse" repo. Not the one in dataverse-ansible.
14:11 pdurbin
I'm not trying to over complicate this.
14:12 pdurbin
I'm just thinking that it would be nice to be able to add sample data to any environment, no matter how it was spun up.
14:12 donsizemore
i'm just aiming for flexibility (and also convenience). yes, you'd point your group_var at your favorite sample data directory (or zipfile, or tarball)
14:14 pdurbin
Sure, I get that it would work fine for Ansible installations. But I don't use Ansible on the phoenix server. I call my birds and trees scripts. Maybe this is all fine. Again, just thinking out loud.
14:16 pdurbin
Right. I call them from scripts/deploy/phoenix.dataverse.org/post
14:17 pdurbin
And I publish the "Spruce Goose" dataset, which has a file. Then from Jenkins I run this test:
14:17 pdurbin
STATUS_CODE_FROM_DOWNLOADING_PUBLISHED_FILE=$(curl --insecure --write-out %{http_code} --silent --output /dev/null https://phoenix.dataverse.org/api/access/datafile/$FILE_ID_OF_A_PUBLISHED_FILE)
14:18 pdurbin
I fail the build if I'm unable to download that published file.
14:18 donsizemore
i do like passing tests
14:18 pdurbin
:)
14:18 pdurbin
Anyway, I don't want to add blockers to what you're doing.
14:19 pdurbin
But would it make sense to create a standalone script instead?
14:19 donsizemore
we could. same variables/directory dealy?
14:20 donsizemore
what i'm hearing is "sample data plugin"
14:21 pdurbin
I mean, in practice, we'll benefit a lot from your original direction. The design team doesn't want a barren installation. That's what they get now when they use the ec2-create script. That script uses Ansible. So if it's easiest, maybe we can prove the concept in Ansible. And worry about extracting it later.
14:22 donsizemore
i can just write it into ansible, it'll be off by default, and if need be there can be some duplication elsewhere
14:22 pdurbin
Sounds perfect.
14:23 pdurbin
I'm planning on keeping my birds and trees scripts around anyway. I like being able to say "here's a curl command to create a dataset that works every time I run the API test suite". curl is more universal than Ansible.
14:23 donsizemore
one of the few things that doesn't go out of fashion every 5 years =)
14:24 pdurbin
Heh. In my day we didn't have Ansible. And we didn't call it DevOps.
14:25 donsizemore
i still have solaris 2.5 installation CDs if you want to borrow them
14:26 pdurbin
Wow, mine only go back to 2.6.
15:18 donsizemore joined #dataverse
15:52 pameyer joined #dataverse
16:37 pdurbin
pameyer: morning. I see you're assigned to https://github.com/IQSS/dataverse/issues/5424 . Need anything?
16:38 pameyer
pdurbin: thanks; don't think so
16:38 pameyer
talked to danny, going to expand scope to fix the capitalization thing in this branch too
16:39 pdurbin
Ok. I just left a review saying that a unit test would be nice.
16:39 pameyer
well, I had to fix the unit tests when I fixed the capitalization
16:40 pdurbin
Ok, I'm not sure about the details but it sounds like you have it well in hand.
16:42 pameyer
that's always debatable ;) it's not top of my list, but mostly done
16:44 pdurbin
Cool. Remember talking about valid SSL certs on EC2 instances created by our "spin up" script? Looks like Slava may have already cracked that nut? Check out https://github.com/IQSS/dataverse-docker/blob/4.9.3/kubernetes/infrastructure/docker/letsencrypt/entrypoint.sh
16:45 pameyer
open source is helpful sometimes
16:45 pdurbin
heh
18:19 donsizemore joined #dataverse
19:25 andrewSC joined #dataverse
20:15 pameyer
has anyone else noticed the installer ending up with glassfish not running?
20:16 pdurbin
not consistently
20:17 pameyer
it's been consistent with docker-aio, and today I noticed it with the normal installer
20:17 pdurbin
yikes
20:17 pameyer
not a hard problem to solve, but it makes some of the automation unhappy
20:18 pdurbin
I bet. You've already looked into solving it?
20:18 pameyer
not yet
20:19 pameyer
would be interesting to know if it's been showing up with the ec2/ansible provisioning
20:19 pdurbin
docker-aio doesn't use the installer, does it? It doesn't use the Perl script anyway, right?
20:19 pameyer
depends. for ITs, it doesn't. for testing the normal install, or for docker-dcm, it does
20:19 pdurbin
right, right, so you're actually testing the installer
20:20 pameyer
very non-systematicly, but correct
20:20 pameyer
if I'd been able to figure out how to use the perl script installer for ITS, I'd be doing that too
20:28 donsizemore
@pameyer anytime that happens to me, it's been systemd not waiting long enough (so not the installer)
20:28 donsizemore
@pameyer have you tried de-icing the storage?
20:28 pameyer
@donsizemore - sadly that didn't occur to me. guess I need some coffee to de-ice my brain ;)
20:29 pameyer
I don't know that this is systemd related - this docker stuff shouldn't be using systemd
20:29 donsizemore
correct
20:30 pameyer
I wondering about the possiblilty of there being a `asadmin stop-domain` without a start, or a `asadmin restart-domain` that doesn't actually restart having crept into things at some point
20:30 donsizemore
is this on a branch or in release
20:31 pameyer
the normal/perl installer just on a branch so far. setting up for ITs, consistent across multiple branches and the most recent release
20:32 pameyer
https://github.com/IQSS/dataverse/issues/5374 is from a while back when I first noticed it
20:32 pameyer
it's still not high priority for me; more making my ears perk up in case it may be a sign of other problems
20:34 donsizemore
could you send me the server.log(s)? (and branch)
20:35 pameyer
sure - I'll do a clean run on develop to simplify things
20:35 pdurbin
pameyer: does `vagrant up` result in a working installation on your branch? Uses the installer in the "dataverse" repo.
20:36 pameyer
pdurbin - I'll give it a try, but there's a good chance my vargrant version is too old for the dataverse vagrant
20:38 donsizemore
"and the next thing you know, you’re at the zoo, shaving a yak, all so you can wax your car."
20:39 pameyer
:)
20:48 pdurbin
I had to upgrade Vagrant and VirtualBox after upgrading to OS X Mojave. I didn't go for VirtualBox 6.0 though. Scared.
21:30 pameyer
my vagrant is too old :( and upgrading it has a high probability of breaking something else, so that needs to stay frozen.
21:31 pameyer
dying glassfish on install turns out to be very weird. installing w\ exec from outside docker results in no glassfish, exec bash and running the same non-interactive install results in running glassfish
21:31 pameyer
so I'd lean towards either docker weirdness, or something sensitive to an interactive shell failing on a non-interactive shell
21:31 pameyer
or other gremlins
21:45 pdurbin
Well, at least there isn't a reproducibility crisis. It fails every time.
21:48 donsizemore joined #dataverse
21:50 pameyer
yeah; failing predicatably is preferable
22:20 pameyer
@donsizemore - server.log had a bunch of exceptions for the oddness I was seeing with the installer, but it doesn't look like any new exceptions. there's actually more in the log when it finishes.
22:20 pameyer
someday maybe we'll get all of the exceptions out of the logs
22:22 pdurbin
would be nice
22:26 pameyer
baby steps in https://github.com/IQSS/dataverse/issues/5446
22:30 pdurbin
yeah, no 500 errors please
22:32 pameyer
one of the nice things about django is that you can configure it to send email anytime there's an uhandled exception
23:00 pdurbin
Sounds good but I don't want email from other people's installations. :)
23:02 pameyer
I have _no_ idea what you're talking about ;)
23:03 pdurbin
Well, there are 36 installations of Dataverse. I don't want email from each one when there's an uncaught exception. Do I?
23:04 pameyer
pdurbin: I ran across https://www.nature.com/magazine-assets/d41586-018-07780-5/d41586-018-07780-5.pdf
23:11 pdurbin
pameyer: yes! Thanks! Did you take a look at my slides? https://docs.google.com/presentation/d/13Va5AwzNKEFX3YmuuxXrrzIidwlWlFxHyl-U4xA5_Lg/edit?usp=sharing
23:11 pameyer
pdurbin: I haven't had a chance yet :(
23:11 pdurbin
ok, there's still time :)
23:12 pameyer
that's what I was hoping
23:14 pdurbin
my talk is on Feb 6: http://osd.mpdl.mpg.de
23:15 pameyer
Feb isn't as far away as it was
23:16 pdurbin
I'm not so sure about this "limit coffee intake" advice.
23:21 pameyer
I'm sure they meant "limit decaf coffee intake" ;)
