Time
S
Nick
Message
07:18 jri joined #dataverse
08:35 jri_ joined #dataverse
12:16 donsizemore joined #dataverse
12:45 Youssef_Ouahalou joined #dataverse
13:48 pkiraly joined #dataverse
14:03 pdurbin joined #dataverse
14:18 pkiraly
pdurbin, I've worked on the API change: https://github.com/IQSS/dataverse/pull/6848
14:19 pdurbin
pkiraly: that was quick! Do you know how to add your issue to our board? Once it has been added, automation will put it in code review.
14:21 pkiraly
pdurbin, yes, I know, but Danny already did it
14:21 pdurbin
awesome
14:30 pdurbin
pkiraly: someone just volunteered to take a look.
14:33 pkiraly
wonderful!
14:44 donsizemore
@pdurbin bad news for danny. with localstack running in docker, a t2.large no longer makes it up the hill
14:45 pdurbin
Back in my day it was a t2.medium.
14:45 donsizemore
@pdurbin i'm testing my hopefully last commits to #140 and the OoM killer zapped Payara
14:46 pdurbin
I went to look at https://github.com/IQSS/dataverse-ansible/issues/140 to see who opened it and it's me!
14:50 donsizemore
@pdurbin updating dataverse-ansible's Vagrant file is on my list though less to do there. just waiting on the go-ahead to switch the role to payara proper. hoping to merge #140 first
14:53 donsizemore
@pdurbin t2.large worked fine for localstack, the integration tests knocked things over
14:54 pdurbin
donsizemore: cool, speaking of Vagrant, I'm just adding a note to ours saying "This broken, please see 6849." I was also thinking of saying "If you like Vagrant, you can try the one over at dataverse-ansible." What do you think?
14:54 donsizemore
fine by me. especially since it sounds like an ad for a knock-off product ;)
14:55 pdurbin
"It git clones in a fraction of the time!"
15:03 pkiraly joined #dataverse
15:03 pdurbin
20% less sugar!
15:06 donsizemore
@pdurbin so, ran the full test suite on payara5 against develop with s3/localstack. one failure: testFileLevelTool1
15:09 pdurbin
donsizemore: not bad!
15:11 pdurbin
donsizemore: do you have a line number?
15:19 donsizemore
@pdurbin the error was 'cannot get property "DisplayName" on null object' but server.log* is not helpful
15:20 pdurbin
Ok. If you fine the line number in ExternalToolsIT.java I have it up. Not sure why it failed.
15:20 pdurbin
find*
15:23 donsizemore
i'm not finding any related strings in the server logs. i have all the testing output i can send if you like
15:24 donsizemore
line 130
15:25 pdurbin
Ok, so the call to UtilIT.getExternalToolsForFile failed. Hmm.
15:25 donsizemore
nothing for that in *log* either
15:26 pdurbin
It calls into a test API I added. The API is not intended for use by users. I wonder why it's different with localstack.
15:27 donsizemore
there are several deadlocks in postgres' log: STATEMENT: UPDATE DVOBJECT SET PERMISSIONINDEXTIME = $1 WHERE (ID = $2)
15:28 pdurbin
oh, yikes, deadlocks are never good
15:29 donsizemore
not really the time to open issues since we're not on payara yet
15:29 donsizemore
i'm doing a 2nd run now to see if i get the same results
15:31 pdurbin
cool
16:20 donsizemore joined #dataverse
16:23 donsizemore
@pdurbin this time we got an error on testFileLevel1 and two failures: ReplaceFileUnpublishedAndBadIds and testForceReplaceAndUpdate
16:24 donsizemore
and about twice as many postgres deadlocks
16:24 pdurbin
Weird. I just picked up https://github.com/IQSS/dataverse/issues/6846 about FilesIT.testForceReplaceAndUpdate
16:24 pdurbin
But I'm off to lunch.
16:24 donsizemore
you should definitely eat lunch first
17:04 jri joined #dataverse
17:05 donsizemore joined #dataverse
17:24 jri joined #dataverse
17:30 nightowl313 joined #dataverse
17:31 nightowl313
hi all ... I hope it's okay for me to ask questions here? I made the mistake on the community group of sending an email with my signature and that is out there for all time now ... wish there was a way to remove that
17:32 nightowl313
is there a reason for creating test credentials with datacite? is it just to have a different prefix? I created the test credentials, and made the required config changes in my DV, so when I publish datasets/dataverses now, it creates the DOI with the new prefix correctly. However, when I click that link I just get typical “DOI prefix not found” error from the datacite site. I tried creating the DOI on the fabric site, but still does not
17:33 jri joined #dataverse
17:34 nightowl313
is there any way to make DOIs created within DV when datasets are published and have them resolve correctly with a datacite test account?
17:35 donsizemore
@nightowl313 welcome! you're more than welcome to ask questions here
17:36 donsizemore
test credentials create test DOIs, which can be thrown away. real DOIs are supposed to last forever
17:37 nightowl313
it doesn't look like the test DOI's are being created, or maybe I'm doing something incorrectly! does it automatically create the DOI in the test account of datacite?
17:37 nightowl313
when a dataset is published?
17:37 nightowl313
Or, do I have to go in and create it?
17:38 donsizemore
i'm digging out our test credentials so i can go in and look. our archivists always dealt with the DOIs
17:38 nightowl313
either way the link on the published item in dv does not resolve...it just goes to datacite and gives a not found error
17:40 pdurbin
nightowl313: thanks for calling in yesterday
17:41 donsizemore
i finally found the place to log in
17:42 nightowl313
thanks for having those calls! I hope to get more involved with things as I learn more!
17:42 pdurbin
nightowl313: you're saying test DOIs don't resolve
17:43 donsizemore
okay. so our test DOIs resolve
17:43 pdurbin
donsizemore: orly
17:43 donsizemore
or at least i can see the URL in fabrica
17:43 pdurbin
The test DOIs on https://demo.dataverse.org don't resolve.
17:43 nightowl313
hmmmmm ... so I added our username and password to domain.xml and I ran the db command to change the prefix ... restarted the server
17:44 jri joined #dataverse
17:44 donsizemore
you're right, the citation DOI doesn't
17:44 nightowl313
is there anything else that needs to be done?
17:44 nightowl313
i can't see the urls in fabrica either
17:44 nightowl313
okay, yea, that's waht ours are doing as well
17:45 donsizemore
so this doesn't resolve https://doi.org/10.33563/FK2/4ZOPY4/KTTDNY but fabrica offers a working URL to the datafile
17:46 jri joined #dataverse
17:49 nightowl313
ah so should we fill in that form with the correct URL for each of the DOI's that we add to DV?
17:49 donsizemore
this should all happen when the DOI is registered. our production DOIs are fine. I'm wondering if DataCite doesn't care because they're test DOIs
17:51 donsizemore
you might ask support
17:51 pdurbin
I'm asking in Slack but who knows. Test DOIs don't resolve on our demo server and everyone has been ok with that over the years. :)
17:52 pdurbin
nightowl313: is there an email you wanted me to delete from the Google Group?
17:55 nightowl313
okay, I have emailed datacite support for more info ... we just need to understand that, too!
17:56 nightowl313
I just sent some messages through the google group with my phone number and other things on there, and I should have removed all that first; doesn't really matter that much but just trying to reduce the amount of findable info about me out there! haha
17:56 donsizemore
Leonid just noted you're using the FAKE provider?
17:57 donsizemore
Dataverse added that for development (DataCite didn't want to give out anonymous access for testing)
17:57 donsizemore
to make Dataverse talk to DataCite you want to set the provider to 'doi'
17:57 pdurbin
nightowl313: maybe the email didn't go through. I don't see it.
17:58 nightowl313
it's just some of the questions I sent through, such as this one: https://groups.google.com/forum/#!topic/dataverse-community/-V5pgusDgHU
17:59 nightowl313
I left my signature on there
17:59 nightowl313
I don't know if we are using the FAKE provider ... I just ran the install and then changed the test credentials for datacite to our own test creds that they set up for us
18:00 donsizemore
i haven't seen the messages either, just checked my spam folder.
18:00 pdurbin
nightowl313: oh, so lots of messages. Hmm. Well, please let us know if you want us to do something with them.
18:01 donsizemore
dataverse-ansible defaults to the FAKE provider (dataverse.doi.provider)
18:01 pdurbin
The FAKE DOI provider is a "no op". It doesn't actually reach out over the wire to a DOI provider. It's for dev and testing.
18:02 nightowl313
na that's okay, I'll just be more careful when I send questinos ... I think I only have 2-3 in there
18:02 donsizemore
not 'doi' DataCite http://guides.dataverse.org/en/latest/installation/config.html?highlight=doiprovider#doiprovider
18:02 nightowl313
to change from the FAKE provider, would I change the info in the rest of the config fields?
18:03 pdurbin
No, it's just a database setting. You can run a curl command.
18:03 nightowl313
oh right, I mean db settings ... AH! I def need to change that!
18:03 pdurbin
http://guides.dataverse.org/en/4.20/installation/config.html#doiprovider
18:04 jri joined #dataverse
18:05 nightowl313
okay, I will change that ... thanks so much for the help! i hope to know enough to provide some help to other orgs at some point, but I'm still in the learning phase, as some of you (Don) know all too well! haha
18:06 nightowl313
this is an incredibly supportive community! thanks for all the help!
18:06 pdurbin
You're quite welcome. :)
18:06 donsizemore
i didn't know my test DOIs didn't resolve!
18:06 pdurbin
nightowl313: any feedback from the pilot for us?
18:08 nightowl313
we have several people following a test document that we set up with a bunch of things to test ... as soon as they have all completed it, I will compile all of the comments/suggestions/issues and post it
18:09 pdurbin
perfect
18:09 nightowl313
some of the things that have already come out of it were things that I needed to fix ... like this DOI setting, and the fact that mail wasn't sending correclty! Don helped me with that!
18:09 nightowl313
I actually ended up setting it up to send via sendgrid
18:09 nightowl313
and it is working great now
18:10 nightowl313
but, I will communicate anything else that comes out of it ... we have about 20 people testing and then will open it up to more across the university
18:10 pdurbin
nice!
18:14 pdurbin
I hope we get your installation on the map some day. :)
18:19 nightowl313
Me too! I feel pretty good that this pilot will turn into a production site and we can add it soon!
18:20 pdurbin
Cool. We have a dot in Texas but we could use some more dots in surrounding states. :)
18:20 pdurbin
https://dataverse.org/installations
18:22 nightowl313
cool! need more of those mid-west and west coast states on there, too! =)
18:23 pdurbin
Yeah, I'm from Ohio. Maybe there's someone I can talk to. :)
18:23 donsizemore
i volunteer to do the tropical installations. no dots in the caribbean, for instance.
18:24 nightowl313
=D
18:24 pdurbin
Sigh. If only we could still dream about travel. I'm canceling a trip to Oregon this summer.
18:25 donsizemore
@pdurbin may i pester you for a favor?
18:25 pdurbin
Sure, but I'm also finishing up looking at poikilotherm's aws pull request.
18:26 donsizemore
heh. my next major task will be rewriting dataverse-ansible to use the python3 installer, and its config file, but i'm finding myself doing it already as develop changes.
18:26 donsizemore
so i was going to ask you to take a look at https://github.com/IQSS/dataverse-ansible/pull/166 before i merge it
18:26 donsizemore
oliver's is definitely more important
18:27 pdurbin
So you use default.config?
18:27 donsizemore
that'll be in the rewrite
18:28 jri joined #dataverse
18:28 donsizemore
which will also make oliver happy as it will simplify his dataverse-k8s life in some way
18:28 pdurbin
Gotcha. In vagrant I use the old default.config and the Perl installer. The format changed.
18:30 poikilotherm
nightowl313 pdurbin donsizemore Test Doi from DataCite are not resolvable via the normal doi resolvers by design. As donsizemore says: you need to have em ephemeral
18:30 poikilotherm
You can resolve them when using the DataCite resolver
18:31 poikilotherm
Iirc I created an issue about that a while ago...
18:33 pdurbin
poikilotherm: I just moved your aws pull request to QA and left this comment: https://github.com/IQSS/dataverse/pull/6837#pullrequestreview-398467141
18:34 nightowl313
poikilotherm, so fill in the URL on datacite in the error form that appears when linking to the DOI?
18:34 pdurbin
donsizemore: now I'm looking at your s3 pull request. It's nice to see that ansible is fixed now.
18:36 donsizemore
@pdurbin at the risk of scope creep... IQSS is actively updating the installation process =)
18:38 pdurbin
donsizemore: yep. I don't see anything objectionable in this pull request. Your descriptions suggests more stuff happening than I'm seeing. I don't see a switch to Payara, for example.
18:39 donsizemore
that's next. i just want to clear the plate before delving into a more hefty rewrite (python3 installer / payara)
18:40 pdurbin
Oh, I see. So you want expand the scope. That's fine. It looks good so far. Do you need anything else from me?
18:40 donsizemore
no, i want to merge it =) then start a new branch to rewrite the installer
18:41 donsizemore
jim suggested a couple things so if you and leonid are happy i'll move on
18:41 pdurbin
cool, go for it, I say
18:44 donsizemore
in the next branch i'll be doing what you did for the documentation: s/glassfish/payara/g and hopefully swap out a bunch of custom ansible for leonid's new python installer
18:51 pdurbin
nice
18:52 pdurbin
Yeah, I've been happily running Payara on my Mac for a little while now. Change is in the air.
18:53 pdurbin
The question is, how far can the MicroProfile Config API get us? Can it reduce the size of the Ansible code? Or reduce the size of the Python installer? Only poikilotherm knows.
18:57 jri joined #dataverse
18:57 jri joined #dataverse
19:06 pdurbin
poikilotherm: your Jakarta EE 8 pull request was just merged.
19:13 jri joined #dataverse
19:18 nightowl313 left #dataverse
19:41 pdurbin
donsizemore: knock knock
19:55 pdurbin
I'm thinking I might need to bump these up:
19:55 pdurbin
-XX:MaxMetaspaceSize=512m
19:55 pdurbin
-XX:MetaspaceSize=25m6
19:55 pdurbin
because I'm getting this: java.lang.OutOfMemoryError: Metaspace
20:04 pdurbin
Ok, I doubled them and I can no longer reproduce the failing test at FilesIT.testForceReplaceAndUpdate mentioned at https://github.com/IQSS/dataverse/issues/6846 but I'm thinking about using that issue to represent "get the API test passing" because I'm not seeing *different* tests failing at https://jenkins.dataverse.org/job/IQSS-dataverse-develop/427/
20:04 jri joined #dataverse
20:27 jri joined #dataverse
20:42 donsizemore
@pdurbin whoops! i sent those to leonid (i think)
20:42 pdurbin
The installer sets one of them.
20:42 pdurbin
But I doubled them both.
20:43 pdurbin
And now I'm seeing a lot of this: org.postgresql.util.PSQLException: ERROR: deadlock detected
20:46 pdurbin
On my Mac, I mean. My laptop.
21:37 pdurbin left #dataverse
22:00 poikilotherm
Woohoo we're now again part of the cool kids gang, using hippie Jakarta EE 8 :-) I'm so exited about all of this!!!!
