Time
S
Nick
Message
02:38 ismagilani joined #dataverse
06:41 jri joined #dataverse
08:04 Youssef_Ouahalou joined #dataverse
11:30 donsizemore joined #dataverse
11:44 Youssef_Ouahalou joined #dataverse
14:21 pameyer joined #dataverse
14:24 pdurbin joined #dataverse
14:25 pdurbin
poikilotherm: did you see this? https://github.com/IQSS/dataverse-docker/releases/tag/5.0.1-cvm
14:25 poikilotherm
Yeah
14:26 pdurbin
Did you see the thread about someone looking for 5.0 images?
14:31 pdurbin
This: https://groups.google.com/g/dataverse-community/c/ehKHPerQtX8/m/TtJKS7dfAAAJ
14:37 pdurbin
pameyer: cool about postgres in Docker but I'm not sure how much I want to push Docker in the dev guide. Right now Docker is only suggested for testing. But maybe someday we could have the "classic" way of setting up a dev environment next to a fancy Docker way. :)
14:42 pameyer
pdurbin: good point - I'd forgotten that you probably want to dogfood the dev guide
14:44 pdurbin
Yeah, that's what I'm doing. And I'm wondering who wrote this. And it was me.
14:44 pameyer
:) that happens to all of us
14:50 donsizemore
"all documentation is obsolete within ___"
14:50 donsizemore
wrong answers only
14:50 pdurbin
I actually didn't write this bit about recommending Oracle Java. It was like that when I got here. These days OpenJDK seems to be the way to go, at least for developers.
14:57 pameyer
I doubt anyone has a production deployment on osx/macos
14:58 pdurbin
very low chance
17:41 pdurbin
https://demo.dataverse.org is now running Dataverse 5.0
18:11 pameyer
cool
18:37 nightowl313 joined #dataverse
18:41 nightowl313
hi all ... wonder if I could ask a question about DV and S3 ... we are trying to move our pilot DV installation to prod in AWS but in going through the install steps again, I am running into issues I think with permissions, specifically with utilizing S3 for storage; it is working great on our pilot site, but I think on that site I had run dataverse as the root user (didn't realize I shouldn't) :-); I followed the doc instructions befor
18:42 nightowl313
now, I am trying to make sure dataverse is running as an unprivileged user, and when i do the s3 option does not show in dataverse settings (the option to choose storage doesn't show at all)
18:42 pdurbin
Which version of Dataverse are you installing?
18:43 nightowl313
5
18:43 nightowl313
but I installed 5 and ran it as root and it worked
18:43 pdurbin
Nice. Me too. Only my new laptop. Well, the "develop" branch.
18:43 nightowl313
i realize running as root is not a good idea, but does running as a different user add a step to the config that isn't in the docs?
18:44 nightowl313
v5 is amazing! we are excited to get it running as a production site ... we finished our pilot and are ready to go!
18:44 nightowl313
we have been waiting for it
18:44 pdurbin
When you say something doesn't show in the settings, where are you looking? JVM options using asadmin?
18:45 pameyer
I think that one of the recommendations is for the app server to not be able to write to code directories - could the issue be that the non-root user needs a permission change for the setting change to take?
18:45 nightowl313
the jvm options are all there, but when I go into the dataverse dashboard as admin user, it is not in the storage options
18:45 nightowl313
and, if I try to save a file when s3 is the default drive in the config, it gives an error
18:46 donsizemore
@nightowl313 are direct upload and download enabled, and if so, is CORS on the bucket?
18:47 pdurbin
I thought the storage options were on each (sub) dataverse, not the dashboard.
18:49 nightowl313
:donsizemore has been immensely helpful in the last few weeks helping me with all of this! .. direct upload and download are set to false
18:50 nightowl313
in the main dashboard for the root dataverse, in the general settings there is a "storage" option and you can switch between all of your available storage options
18:50 nightowl313
that is when I run as root
18:50 pdurbin
Ok, each dataverse should have that.
18:50 nightowl313
I have a site setup that is running root, and I am trying to install a different one running as the payara (dataverse) user
18:51 pdurbin
The site running as root is also 5.0?
18:51 nightowl313
i see it in the general settings of each dataverse as well
18:51 nightowl313
yes they are both v5
18:52 pdurbin
very strange
18:52 donsizemore
s3 doesn't write very much to server.log, i've found
18:52 pdurbin
I mean, they're just JVM options, I think.
18:53 pameyer
does configuring other jvm options as the payara/dataverse user work?
18:53 donsizemore
@nightowl you have -Ddataverse.files.storage-driver-id=s3
18:53 donsizemore
@nightowl and -Ddataverse.files.s3.type=s3 ?
18:56 nightowl313
yes the options that are in there are: s3.type=s3, s3.label=s3, s3.bucket-name=<bucket name>, s3.download-redirect=false, s3.upload-redirect=false, s3.url-expiration-minutes=60
18:56 nightowl313
thought I deleted that last one
18:56 nightowl313
:q!
18:56 donsizemore
and you've restarted payara since any such changes, and you can 'aws s3 cp' a test file into that bucket
18:57 donsizemore
it may be time to summon qqmyers
18:57 pdurbin
Heh. I just sent him a message on Slack. :)
18:57 nightowl313
yes, was able to successfully copy a file up to the bucket from cli
18:57 Jim15 joined #dataverse
18:57 donsizemore
do you get much in server.log? to my experience you get a brief "network error" warning in the web interface, which disappears
18:58 pdurbin
Jim15: thanks for joining us!
18:58 donsizemore
the spell worked!
18:59 Jim15
hi all
18:59 pdurbin
How can we summarize? nightowl313 has two almost identically configured installations of Dataverse 5. One is running Payara as root and correctly shows a dropdown between storage options. The non-root installation does not show the dropdown. Is that right?
19:00 Jim15
the only thing I can see from a quick run through the chat log is to suggest making sure that the user running payara has the aws config in its ~/.aws dir
19:00 nightowl313
yes
19:00 nightowl313
the aws config is there and I can list the bucket contents from the cli
19:00 nightowl313
when logged as that user
19:01 Jim15
The dropdown to select a store in the UI is only for superadmins, so it depends on who you log in as...
19:01 nightowl313
in the server log .. there is a message "Failed to look up bucket <bucket name> (is AWS properly configured?))|#]"
19:01 donsizemore
@Jim15 I had her 'aws s3 cp' a test file into the bucket, that succeeded
19:02 Jim15
That suggests it can't get the aws credentials - it's basically the first thing that's tried.
19:02 donsizemore
if this were me, it would be a typo in the bucket name.
19:02 Jim15
how about the profile - do you only have a default aws profile / or is dataverse configured to get the right one for the bucket
19:03 pdurbin
donsizemore: pameyer is the one who always calls himself typoman
19:03 nightowl313
there is only one in there on the server ... it has full s3 privs (which it probably shouldn't)
19:04 Jim15
it doesn't help today but for QDR I added a line at that 'Failed' message to also print the full message from AWS about what went wrong. I'll try to remember to push that to develop...
19:04 pameyer
yup - more typos some days than others
19:04 pameyer
I'm out of the loop with the s3 stuff; but could selinux be a factor worth checking?
19:05 donsizemore
disabled i think
19:05 nightowl313
i also had issues with ssl which work fine using the root user ... so just think there is some permission thing with running as the unpriv user
19:05 pdurbin
Jim15: sounds like a useful line :)
19:05 nightowl313
oops I disabled it on the instance that I blew away last night ... didn't on this one ... will try it
19:06 nightowl313
shout out to donsizemore (again) for his longsuffering help ... just felt so bad that I kept bugging him .. so I came here
19:06 donsizemore
the iRODS group has a technical problem Bingo card on their whiteboard. more of a histogram, really
19:06 donsizemore
SElinux had a bunch of marks next to it
19:06 pameyer
:)
19:06 donsizemore
@nightowl313 it's been good for me - you've found so many bugs in dataverse-ansible!
19:07 nightowl313
haha .. I've run it at least 100 times ... it is magical!
19:08 donsizemore
the thing is, ansible will configure your instances to install and run payara as the dataverse service account
19:09 donsizemore
so when you're running dataverse as root, is that a manual command? ansible shouldn't be writing the s3 credentials under the root user.
19:09 nightowl313
we are using dataverse-ansible in our disaster recovery procedures ... I would not have gotten this all working without it
19:10 nightowl313
disabled selinux, rebooted server, still can't upload files via s3
19:10 pdurbin
bah
19:10 donsizemore
ok. so payara is running as user 'dataverse', and you as 'dataverse' at the cli can upload a file to the bucket
19:12 nightowl313
yes ... logging in as user dataverse .. i can do aws s3 commands on the bucket
19:12 donsizemore
with direct-upload and direct-download disabled (just so everybody's caught up)
19:12 Jim15
If it isn't showing up in the UI for a superuser (Dataverse/General Info), that would suggest typos in the jvm-options - that UI is populated from the options without trying to access AWS.
19:13 donsizemore
it me
19:14 nightowl313
typos are definitely possible ... let me check them carefully ... the ansible script adds them but I removed them all and added them manually
19:15 nightowl313
but i copied the code from the docs
19:16 nightowl313
./asadmin create-jvm-options "-Ddataverse.files.<id>.download-redirect=false"
19:16 nightowl313
./asadmin create-jvm-options "-Ddataverse.files.<id>.upload-redirect=false"
19:17 donsizemore
you want that <id> to be s3
19:17 nightowl313
oh yea, sorry, I did change that :-)
19:17 Jim15
-Ddataverse.files.s3.label=yourLabel and -Ddataverse.files.s3.type=s3 are the key ones - if they exist I think you'll see a store listed, even if the bucket isn't set up
19:17 donsizemore
though i see from your paste above you have them
19:18 pameyer
have you confirmed that the jvm options are persisting to domain.xml?
19:19 donsizemore
wait wut
19:19 nightowl313
here they are directly from my config file: <jvm-options>-Ddataverse.files.s3.type=s3</jvm-options>
19:19 nightowl313
<jvm-options>-Ddataverse.files.s3.label=s3-aws</jvm-options>
19:19 nightowl313
<jvm-options>-Ddataverse.files.s3.bucket-name=dataverse-test-user-files</jvm-options>
19:19 nightowl313
oops that's my bucket name
19:19 nightowl313
<jvm-options>-Ddataverse.files.s3.url-expiration-minutes=60</jvm-options>
19:19 nightowl313
<jvm-options>-Ddataverse.files.storage-driver-id=s3</jvm-options>
19:20 nightowl313
<jvm-options>-Ddataverse.files.s3.download-redirect=false</jvm-options>
19:20 nightowl313
<jvm-options>-Ddataverse.files.s3.upload-redirect=false</jvm-options>
19:20 pameyer
good; one more thing ruled out
19:20 donsizemore
all of this looks right
19:20 nightowl313
there is not a jvm option for storage-driver-id=files
19:22 nightowl313
i deleted it
19:22 Jim15
that just sets the default store - I think it still defaults to file if not set. If you set it to s3, that would make s3 your default.
19:23 Jim15
yeah - with the other options, I don't see anything
19:23 nightowl313
do some installations run as root?
19:23 nightowl313
i assume that's a bad thing :-)
19:23 donsizemore
it _could_ be a bad thing
19:23 nightowl313
we have been running our pilot one as root without realizing it ... i am learning all of this as I go (obviously)!! :-D
19:24 nightowl313
and with don's help!
19:24 donsizemore
running as root is like driving without a seat belt. 99% of the time it's fine
19:25 nightowl313
yea, I really don't want to do that if possible ...
19:25 pdurbin
Friends don't let friends run Dataverse as root.
19:25 nightowl313
haha
19:25 pameyer
if it listens on the internet, it probably shouldn't be running as root
19:25 Jim15
So you don't see any UI options for the store when you login as dataverseAdmin?
19:26 nightowl313
we are hoping to announce our live production site soon ... and will soon want our own dot on the map :-)
19:26 pdurbin
mmm, dot
19:26 nightowl313
yes, nothing ... there is a blank space between affiliation and description
19:26 nightowl313
on the harvard site dataverse installation map
19:28 pdurbin
well, it's the community map, really :)
19:28 nightowl313
re: the "dot" for that last comment
19:29 pdurbin
Jim15: you're saying the UI options are purely driven by those JVM options?
19:29 donsizemore
@Jim15 I'm playing along at home; under 5.0 I have both files, my bucket, and my bucket (default)
19:30 Jim15
yes. and even when I only have one store, I see two options (file(default) and file)
19:30 Jim15
(of those options, the first will change if you change that storage-driver-id jvm option and the latter would force the dataverse to stay with file)
19:31 nightowl313
that is how it shows on the installation running as root on mine
19:31 nightowl313
i even added a wasabi storage option
19:31 donsizemore
@nightowl313 is there a rivalry with that "other" school in Tucson? cause i'm impressed https://twitter.com/cfishman/status/1299049476288544768
19:31 nightowl313
omg
19:32 nightowl313
waste water testing? is that real?
19:32 donsizemore
I wish UNC had done that three weeks ago
19:32 nightowl313
i thought u of a decided not to have on campus classes? ASU definitely IS!
19:33 donsizemore
on running dataverse unprivileged - there's no chance that the current payara installation could've been launched as root during testing?
19:33 Jim15
So without the UI to change, how did you hit the error about Failing with S3? Are there existing files in S3?
19:34 nightowl313
i have it as the default ... so i believe it is just trying s3 when I upload files no matter what?
19:34 Jim15
storage-driver-id=s3?
19:35 nightowl313
i don't think I launched payara as root ... I issued "sudo systemctl start payara" though
19:36 nightowl313
otherwise, have used "sudo -u dataverse /usr/local/payara5/bin/asadmin start-domain"
19:36 donsizemore
each of those should be fine
19:36 donsizemore
if you look in /usr/lib/systemd/system/payara.service you'll see it launching as dataverse (I hope)
19:37 nightowl313
yes
19:38 donsizemore
just for a quick test. stop payara
19:39 donsizemore
sudo -u dataverse rm -rf /usr/local/payara5/glassfish/domain/domain1/generated
19:39 donsizemore
sudo -u dataverse rm -rf /usr/local/payara5/glassfish/domain/domain1/osgi-cache
19:39 donsizemore
then start payara
19:40 nightowl313
okay payara is restarting ...
19:41 nightowl313
storage options still missing :-)
19:41 donsizemore
can you send me the server.log from this most recent launch?
19:42 Jim15
@pdurbin - fwiw: confirmed that just having the type and label jvm options set are enough to have a store show in the UI
19:43 Jim15
and you've logged in as dataverseAdmin?
19:43 nightowl313
yes, logged in as dataverseAdmin ... DS will send my log ... thanks everyone for the help!
19:44 Jim15
problems are fun when they're not yours :-)
19:45 pdurbin
Jim15: thanks for checking
19:45 donsizemore
i wanna know if ansible is somehow responsible
19:45 Jim15
and just to confirm - if you go to the dashboard (menu under your account) and the manage users button, you see dataversAdmin with the Superuser box checked?
19:48 nightowl313
i don't see Dashboard as an option! dataversAdmin must not be superuser! Did NOT notice that!
19:48 Jim15
cool - I was having a hard time coming up with one problem that could cause both symptoms.
19:50 nightowl313
doesn't it become superuser in the installation?
19:50 Jim15
it should
19:50 donsizemore
curl -X POST http://localhost:8080/api/admin/superuser/dataverseAdmin
19:53 donsizemore
that used to happen in setup-all.sh
19:54 donsizemore
oh. https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/blob/0225a507ff6e759421ae4cc4cca8f2b9131d75ef/tasks/dataverse-postinstall.yml#L28
19:55 donsizemore
but i don't see the python installer doing that.
19:56 pdurbin
If it helps, the name is red when it's a superuser.
19:56 nightowl313
so, the option is there for storage, dataverseAdmin is superuser, but still can't upload files via s3 :-(
19:56 nightowl313
if I change it to files it works great
19:56 pdurbin
sounds like progress! at least the option is there :)
19:57 nightowl313
yes!!! yay!@
19:57 donsizemore
yes, the python installer does run setup-all.sh https://github.com/IQSS/dataverse/blob/develop/scripts/installer/install.py#L580
19:57 donsizemore
so how is your dataverseAdmin not a superuser?
19:58 nightowl313
???
19:59 donsizemore
no SEVEREs on startup
19:59 donsizemore
and absolutely nothing about s3.
19:59 donsizemore
is there a setup-all.out in /tmp/dvinstall or /tmp/dataverse?
20:01 nightowl313
yes
20:01 donsizemore
could you send me that as well?
20:01 donsizemore
brb
20:02 nightowl313
yes, there are alot of database errors in it
20:03 Jim15
FWIW: w.r.t. the Failure message above - the latest develop branch includes the more detailed message from the AWS client in the log
20:11 donsizemore
those database errors are normal - they all end in "already exists"
20:11 donsizemore
though WARNING: failed to import reference data!
20:12 donsizemore
there's not much echoed after "Running post-deployment setup script (setup-all.sh)" (and no errors at least)
20:12 donsizemore
Jim, is there any requirement of a DataCite account for S3 (publishing/reservation?)
20:13 donsizemore
oh, never mind. i see them further up in the log.
20:14 donsizemore
@nightowl313 it looks like you're re-running the installer on this instance. there's no chance the DB is from the 4.20 era?
20:16 donsizemore
that shouldn't make a difference either, just askin
20:18 donsizemore
@nightowl313 setup-all did it =) "{"status":"OK","data":{"message":"User @dataverseAdmin removed as a superuser."}}"
20:18 donsizemore
that API call is a toggle, and on successive runs if dataverseAdmin exists, setup-all crabs then calls the superUser endpoint anyway
20:19 donsizemore
then root dataverse, metadatablock and facet steps fail on bad API key, but if this is a re-run they should exist anyway
20:19 donsizemore
speaking of re-run, i'm going to have to run a couple errands, will check back in for more logs later?
20:22 pdurbin
I hate that toggle.
20:27 pameyer
that does seem an odd place to have that behavior
20:28 nightowl313
i reran the installer a few times because there were some errors, but it is a brand new instance anda everything is v5
20:32 pdurbin
If it makes you feel any better, I'm still struggling to get Dataverse 5 installed on my laptop.
20:32 nightowl313
that does make me feel better! haha ... trying so hard to learn how this all works but there is a lot to it! so grateful for the help
20:32 andrewSC joined #dataverse
20:33 nightowl313
so ... I'm going to create a new instance and try this all again ... and try to only run the installer once ... lol
20:33 pdurbin
I'm stuck on "Couldn't create database or database already exist"
20:33 pdurbin
exists*
20:34 nightowl313
is there any easy way to just wipe everything out and start over? I typically just blow away the instance and create a new one .. can't really run this in a vm or vagrant because of all the specific IPs and urls, etc.
20:34 nightowl313
other than uninistalling everything
20:36 pdurbin
Well, this is on my Mac. I've tried dropping the database and user.
20:37 pdurbin
nightowl313: for servers I wrote this (a while ago): http://guides.dataverse.org/en/5.0/installation/installation-main.html#fresh-reinstall
20:39 nightowl313
oh yea, sure, I have done those steps and will do that ... was just looking for a completely clean slate ... but prob not necessary
20:39 nightowl313
thanks!
20:41 pameyer
nightowl313: this may be too late this time, but this is a case where having snapshots of the VM you're working on is really helpful
20:42 pdurbin
Huh, might be my problem: source database "template1" is being accessed by other users
20:44 pdurbin
yeah, that was it. weird
20:45 nightowl313
other users on your mac?
20:46 pdurbin
no, just me in another terminal window :)
20:46 pdurbin
but weird that the installer cares
20:47 pdurbin
anyway, I've got 5.0 running now. phew :)
20:49 nightowl313
are you using s3 storage?
20:50 pdurbin
Nope. I've done a tiny bit of dev on S3 stuff so I had it configured on my old laptop.
20:58 pdurbin
Anyway, I'm heading out. Have a good evening, all.
20:58 pdurbin left #dataverse
21:07 nightowl313
bye! thanks so much for your help!
21:22 nightowl313 left #dataverse
22:41 Jim15 left #dataverse
