Time
S
Nick
Message
07:42 Virgile joined #dataverse
09:54 sonja18 joined #dataverse
12:07 donsizemore joined #dataverse
12:41 juancorr joined #dataverse
15:19 dataverse-tubs joined #dataverse
15:22 dataverse-tubs
I tried to upload a new metadata block and tried as an example the block dataverse/scripts/api/data/metadatablocks/customPSRI.tsv via curl. I get back {} as an answer. Unfortunately the block is not available in the UI.
15:23 pdurbin
{} as an answer isn't very helpful. :( Is there any detail in server.log?
15:33 dataverse-tubs
mhm `[2020-11-24T16:32:30.712+0100] [Payara 5.2020] [SEVERE] [] [edu.harvard.iq.dataverse.api.errorhandlers.ThrowableHandler] [tid: _ThreadID=68 _ThreadName=http-thread-pool::http-listener-1(4)] [timeMillis: 1606231950712] [levelValue: 1000] [[ _status="ERROR";_code=500;_message="Internal server error. More details available at the server logs.";_incidentId="10ee8ebf-3f38-4320-be21-ef3f369243a6";_interalError="IOException";_request
15:34 dataverse-tubs
is there any other log file then server.log?
15:42 dataverse-tubs
I uploaded with `curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file /tmp/new-metadata-block.tsv`
15:42 dataverse-tubs
And I am using Dataverse 5.0
16:06 pdurbin
dataverse-tubs: ah, thanks for opening https://github.com/IQSS/dataverse/issues/7437
16:07 dataverse-tubs
(y)
16:08 pdurbin
dataverse-tubs: so you're trying this one? PRSI? https://github.com/IQSS/dataverse/blob/v5.2/scripts/api/data/metadatablocks/customPSRI.tsv
16:28 Fishmongering left #dataverse
16:39 nightowl313 joined #dataverse
16:44 nightowl313
hi all ... in the ongoing saga of moving datasets ... does anyone know if metrics can be migrated to a new dataset? I see the api to get metrics out, but just wondering if there is a way to move them to the new location with the new dataset?
16:45 pdurbin
Are you using "classic" metrics or MDC metrics?
16:48 nightowl313
this is for a dataset moved (manually at this point) from harvard to asu dataverse ... i think we would just need the classic metrics (ie: downloads, views, etc.)
16:50 nightowl313
i don't think the get metadata api includes those, right? i am working on using pyDataverse to move the dataset itself, but I don't see metrics in there ... it's okay if not, just getting all the info
16:51 pdurbin
Right, there's no API to retrieve classic metrics, unfortunately.
16:52 pdurbin
And on the ASU side are you using classic or MDC metrics?
16:52 nightowl313
we do have MDC installed but haven't done much with it yet ... could we associate them with the new dataset?
16:53 nightowl313
the ones from harvard that is
16:54 pdurbin
Well, with either system it's a matter of creating rows in some tables.
16:54 pdurbin
There's no automated process for this when moving datasets from one installation to another.
16:55 pdurbin
As we're discovering, moving datasets is not especially well supported. There will be some hacking involved.
16:55 nightowl313
ah okay ... no worries .. i can look into adding rows to the table at some point ... i don't mind doing that as long as it doesn't cause other issues with the database =)
16:56 pdurbin
The MDC tables at least are pretty independent.
16:57 nightowl313
oh yea, no problem ... I am trying to use pyDataverse (thanks to Jim M help) to move a dataset and I will document that process as best I can if it helps at all and if I can get it to work!
16:58 pdurbin
awesome
16:58 nightowl313
BTW ... I was able to create a /guides directory in my docroot and place an html file there and link it in our dataverse! I had to create custom CSS for it, but it worked!
16:59 pdurbin
nice
16:59 nightowl313
I had used dataverse-ansible to install/configure apache so I think it was mostly already configured
17:00 pdurbin
it's magic
17:01 nightowl313
haha I really think it is! ... a "work of art"!
17:04 * pdurbin
gives donsizemore the blue ribbon
17:07 nightowl313
:)
17:08 nightowl313
... he probabaly has nightmares about helping me with dataverse-ansible!
17:12 pdurbin
cold sweats
17:12 nightowl313
:D
18:56 pameyer joined #dataverse
19:54 pameyer
possibly more of a matter of opinion, but is there a level of cpu usage for payara/glassfish in production where it's concerning if it's over that for too long?
19:57 pdurbin
Uh oh. Are you running a fever?
19:57 pameyer
fortunately, still doing load testing
19:58 pameyer
then realized that I didn't know how much additional load would be problematic
20:03 pdurbin
Hmm. Maybe get a baseline and then load test.
20:08 pameyer
maybe - I've usually been pretty lucky with load tests. run a test, notice cpu "spikes", realize the peak of the spike is 1% and call it good
20:38 pdurbin
you call that a spike
20:43 pameyer
yeah - but it means that when I see 25-30%, I realize I need to actually think about it
20:47 pdurbin
It's been a while since we estimated this issue but it contains some thoughts on performance testing: https://github.com/IQSS/dataverse/issues/4201
20:54 pameyer
yeah, that was a while back
20:56 pameyer
performance testing and benchmarking kinds of things can get a little involved sometimes
21:03 nightowl313 joined #dataverse
21:03 pdurbin
My understanding is that we do some manual perf testing before every release.
21:14 poikilotherm
Thanks for updating my branch pdurbin and pushing to code review. There's a button missing for this in the GitHub Android App
21:15 pdurbin
Oh. I did it from the command line. Is there a button in the GitHub web UI for this? :)
21:28 pdurbin
poikilotherm: do you know why "fire off installer" is failing in your branch? fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/bin/python /tmp/dvinstall/install.py...
21:29 pdurbin
I'm looking at https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-7422/5/consoleFull
21:42 pdurbin
Same error with build #6 which was triggered by me updating that branch.
21:48 poikilotherm
Donsizemore might need to address sth in Dataverse Ansible now that the driver is inside the WAR
21:48 poikilotherm
It looks like ansible failed to run
21:49 pdurbin
poikilotherm: do you minding opening an issue in https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible ?
21:49 pdurbin
mind*
21:50 pameyer
is /tmp from the jenkins runs accessable?
21:50 pdurbin
not sure
21:56 pameyer
me either - but looked to me like that's where the logs are going
21:56 pameyer
seems strange to me that the installer failing would result in a successful pipeline, but that might just be me
21:57 pdurbin
:)
22:00 poikilotherm
https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/issues/93#issue-750098489
22:00 poikilotherm
https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/issues/93#issue-750098489
22:02 poikilotherm
Dataverse-ansible is applying patches to the installer. Maybe things go wrong there...
22:04 pameyer
I'd guess there'a already an issue for making the installer configurable enough it doesn't need patches for ci
22:06 pameyer
"there's", I mean
22:18 pdurbin
Thanks for opening that issue. Gotta run.
22:18 pdurbin left #dataverse
