Time
S
Nick
Message
02:09 axfelix joined #dataverse
02:10 axfelix joined #dataverse
03:45 pdurbin joined #dataverse
03:51 yoh joined #dataverse
04:42 axfelix joined #dataverse
07:12 yoh joined #dataverse
07:12 jeffspies______ joined #dataverse
07:12 JonathanNeal joined #dataverse
07:12 dzho joined #dataverse
07:12 andrewSC joined #dataverse
07:12 bjonnh joined #dataverse
07:12 sivoais joined #dataverse
07:12 bret joined #dataverse
07:12 karissa joined #dataverse
08:57 jri joined #dataverse
13:49 donsizemore joined #dataverse
13:56 rebecabarros joined #dataverse
14:10 rebecabarros
pameyer: I was away for a few days but I just saw your answer for my last question in the chat log. I ran the script from '/ansible/roles/dcm/files/root/scripts/dcm-test01.sh' and I've got only: format is: dcm-test01.sh [test dataset id] [test user id]
14:10 pdurbin
rebecabarros: hi! That means you need to give that script two arguments, I believe.
14:11 pdurbin
Yeah, two arguments. I just looked at https://github.com/sbgrid/data-capture-module/blob/master/ansible/roles/dcm/files/root/scripts/dcm-test01.sh
14:12 pdurbin
rebecabarros: does that make sense?
14:13 rebecabarros
pdurbin: yes, it does. Thanks. In that case, dataset and user id from my Dvn, right? I have to look in the database. I will try that
14:14 pdurbin
rebecabarros: to be honest, I'd need to look it up in my tests. One sec.
14:16 pdurbin
That script is calling "ur.py" which means "upload request".
14:17 pdurbin
On the Dataverse side we have a line to log this: "User id " + authenticatedUser.getId() + " had a problem retrieving rsync script for dataset id " + dataset.getId() + " from Data Capture Module."
14:18 pdurbin
rebecabarros: so yes, you would pass the database id of the dataset and the database id of the authenticated user. These are both numbers.
14:18 pdurbin
rebecabarros: that said, can you please remind me where you are having trouble?
14:20 rebecabarros
pdurbin: yes. I've finished the DCM installation and I've change the Dataverse upload mode to dcm. But I'm getting ' Rsync script not available!' from the GUI and {"status":"ERROR","message":"Something went wrong attempting to download rsync script: When making the upload request, rather than 200 the status code was 500. The body was '<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional/
14:22 pdurbin
rebecabarros: Why is there a 500 error? What does the stack trace in server.log say? Do you want to email server.log to support
14:30 rebecabarros
Let me try run dcm-test01 first, to see what will be the output.
14:31 pdurbin
ok
14:32 pdurbin
donsizemore: mornin'. I just left you a comment about the installer at https://docs.google.com/spreadsheets/d/1pl9U0_CtWQ3oz6ZllvSHeyB0EG1M_vZEC_aZ7hREnhE/edit?usp=sharing
14:39 rebecabarros
dcm-test01 returns 403 - Forbidden
14:41 pdurbin
rebecabarros: this is a test installation right? You might find this helpful: 403 - Forbidden: allow Dataverse to talk to DCM · Issue #10 · sbgrid/data-capture-module - https://github.com/sbgrid/data-capture-module/issues/10
14:43 rebecabarros
right. I will take a look. Thanks, pdurbin.
14:45 pdurbin
rebecabarros: awesome. Please feel free to leave a comment on that issue.
14:45 pdurbin
donsizemore: I was confused by Jon saying "I recall a community group meeting where a group was formed to begin exploring collaborations around this area." It has finally dawned on me what he was talking about. He has a good memory. :)
15:24 pameyer joined #dataverse
15:30 pameyer
rebecabarros: if you're only testing the DCM (which is what the test scripts are designed for), you don't need to have it connected to a DV installation
15:30 pameyer
aka - the dataset id and user id can be arbritrary
15:30 pameyer
and if I'm remembering correctly, the user id is a hold-over from earlier design and may not be necessary
15:31 pdurbin
pameyer: mornin'. Do you think she's suffering from that issue I linke about the 403 Forbidden? I hope Im not steering her wrong.
15:31 pameyer
not sure - I'm wondering where the xml is coming from
15:31 pameyer
I don't *think* lighttpd will send that for error cases
15:32 pdurbin
pameyer: me too. I asked her to email her server.log to support
15:32 pameyer
I've only got 10G of email so far ;)
15:33 pdurbin
Heh. Well, please think about how the Data Capture Module should be supported.
15:34 pdurbin
pameyer: it's a specific type of XML . It's XHTML . Probably because it's a 500 error. It would be better if an error were returned in JSON format.
15:35 pameyer
yeah - but is the default mime-type for lighttpd error pages xhtml?
15:35 pameyer
I'm not sure
15:36 pdurbin
No way. Only Dataverse returns XHTML . Everyone else uses HTML5. :)
15:36 pameyer
I *wish* Dataverse returned xhtml
15:36 pdurbin
heh
15:37 pdurbin
pameyer: the XHTML error only gets returned if she uses the Dataverse GUI , not if she hits the DCM directly.
15:38 pameyer
ah - I was wrong, default lighttpd config does have xhtml error pages
16:09 pameyer joined #dataverse
18:08 jri joined #dataverse
18:21 pameyer joined #dataverse
18:36 pameyer joined #dataverse
18:48 jri_ joined #dataverse
18:50 jri joined #dataverse
18:53 jri joined #dataverse
18:57 jri joined #dataverse
19:07 jri joined #dataverse
19:16 jri joined #dataverse
19:17 pameyer joined #dataverse
19:47 UCB_DVN joined #dataverse
19:53 rebecabarros joined #dataverse
19:57 rebecabarros
hey guys, i'm back. so, everything right in /etc/lighttpd and I've notice that in my DCM directory I already have the directory structure from DVN with the cached files from metadata export options
19:59 pdurbin
Sounds like progress.
19:59 rebecabarros
https://pastebin.com/Xx3vyhbH - here it is the dataverse log after I try to download rsync scripts by dataverse api. If you think is better I could send this to the support email
20:01 pameyer
do you get the 500 error when running dcm-test01 from the dv server?
20:02 rebecabarros
should I ran dcm-test01 from the dvn server? I was trying to run in DCM server
20:04 pameyer
it should work on both - but it it doesn't work from the dvn server, then let's you eliminate the DV app as the source of the error
20:06 * pdurbin
hopes Dataverse is innocent
20:09 Thalia_UM joined #dataverse
20:09 rebecabarros
and my domain1/files directory on dataverse side has the same structure of dcm side. meaning, UPLOAD_DIRECTORY and HOLD_DIRECTORY are identical.
20:09 Thalia_UM
Hi Philip
20:09 Thalia_UM
I have a questio
20:09 Thalia_UM
questiona
20:10 Thalia_UM
about XML files
20:10 pameyer
rebecabarros: identical doesn't mean they're the same directory, right?
20:10 Thalia_UM
and SWORD API
20:11 rebecabarros
valid question: my files before change Dataverse to rsync are moved? to where?
20:11 Thalia_UM
SWORD API is for I to add a dataset via curl?
20:12 pdurbin
Thalia_UM: Hi! rebecabarros and pameyer are talking at the moment. Let's let them finish. Thanks.
20:12 rebecabarros
pameyer: identical as with the same directory structure. One requests folder, one processed folder and the corresponding folder from the dataset that i've created after change to rsync
20:13 Thalia_UM
Yes
20:13 Thalia_UM
Thank you
20:13 pameyer
rebecabarros: ok, sounds good
20:18 pdurbin
Thalia_UM: there are two ways to create datasets via API . You can use XML and created datasets via SWORD. Or you can use JSON and create datasets via the "native" API.
20:20 Thalia_UM
yes i read it
20:25 Thalia_UM
the institution that will harvest the dataverse repository asks us for specific metadata, for example:
20:25 Thalia_UM
<dc: creator id = "info: eu-repo / dai / mx / orcid /" 1545-0000-0000-1231> GABRIEL MARTINEZ GARCIA </ dc: creator>
20:25 Thalia_UM
but that as I put it from SWORD, how can I attach the orcid from the metadata?
20:26 Thalia_UM
We from dataverse must consume web services that provided us
20:27 pdurbin
Thalia_UM: Right. I forget. Did you ever email support
20:30 Thalia_UM
I do not send mail about this :-$
20:32 pdurbin
Thalia_UM: ok. The reason I'm asking is that I'm confused. I'm confused because I'm forgetting what we talked about. I forget if Dataverse is harvesting from another system or if another system is harvesting from Dataverse. If you're harvesting, you should use OAI-PMH rather than SWORD or the native API .
20:32 pameyer
I had the vague impression that the goal was to have some information from another web service get incorporated into the dataset when it was created
20:33 pdurbin
yeah
20:36 rebecabarros
pameyer: how do I try to run dc-test01 from my dvn server?
20:38 pameyer
rebecabarros: copy the script over, set DCM_SERVER to your DCM, and run the same way as on the DCM server
20:39 Thalia_UM
I'm also confused because my colleagues who are also working with dataverse, they are using OAI-PMH but they also use SWORD and they want metadata tags to have more attributes like the ORCID in the author and things like that
20:41 Thalia_UM
What is that "@path/to/atom-entry-study.xml" ??
20:42 Thalia_UM
@path/to
20:43 rebecabarros
pameyer: when I try from dvn server I've got 500 - Internal Server Error too
20:45 rebecabarros left #dataverse
20:45 pameyer
rebecabarros: great - narrowing things down. assuming that dcm-test01 works on the DCM server, the problem is probably either firewall or (more likely) lighttpd config
20:46 Thalia_UM
it shows me the following error: curl: (35) gnutls_handshake () failed: The TLS connection was non-properly terminated.
20:52 pdurbin
Thalia_UM: if we are all confused, let's slow down and decide which direction to go. :) I'm a little distracted at the moment so I think it would be better for you to write a few paragraphs about your goal. If you don't want to email supporthttps://groups.google.com/forum/#!forum/dataverse-community . I think a few paragraphs would be better
20:52 pdurbin
than one line at a time here.
20:53 pdurbin
Thalia_UM: Does that make sense?
20:53 Thalia_UM
Yes
20:53 pdurbin
awesome
20:53 Thalia_UM
:-[
20:54 pdurbin
:-[ ? What does that mean? :)
20:55 Thalia_UM
Interoperability with OAI-PMH
20:55 Thalia_UM
Dataverse (institutional repository) will be harvested by a national repository
20:55 Thalia_UM
Correct syntax in metadata with simple DC and OpenAire data guidelines 2.0
20:56 Thalia_UM
That you have the reason to go step by step, because also at the last I do not know what to think or do.
20:57 Thalia_UM
Use some api like SWORD
20:57 Thalia_UM
Consume open web services
21:00 axfelix joined #dataverse
21:03 pdurbin
Thalia_UM: it sounds like you need to create datasets in Dataverse. These datasets will each get a new DOI. I hope that's ok. Then, you will have a national repository harvest from Dataverse. To create datasets in Dataverse you should use JSON and the native API because SWORD is limited in the number of metadata fields you can use.
21:04 Thalia_UM
Yes
21:05 pdurbin
Thalia_UM: here's an example of creating a dataset using JSON and the native API : https://github.com/IQSS/dataverse/blob/v4.8.1/scripts/search/tests/create-all-and-test#L10
21:05 Thalia_UM
Good summary, I like it
21:05 axfelix joined #dataverse
21:05 pdurbin
curl -X POST -H "Content-type:application/json" -d @scripts/search/tests/data/dataset-finch1.json "http://localhost:8080/api/dataverses/finches/datasets/?key=$FINCHKEY "
21:06 Thalia_UM
But What is that line?? @scripts/search/tests/data/dataset-finch1.json
21:06 pdurbin
That line is for the JSON file. It's here: https://github.com/IQSS/dataverse/blob/v4.8.1/scripts/search/tests/data/dataset-finch1.json
21:15 Thalia_UM
What is that ?key=$FINCHKEY ?
21:16 pdurbin
That's an API token.
21:16 pameyer
and it should be a -H "X-Dataverse-key: $FINCHKEY" ;)
21:17 pdurbin
yeah, "?key=" is the old way. Please see https://github.com/IQSS/dataverse/issues/4254
21:19 dataverse-user joined #dataverse
21:19 Thalia_UM
I work with dataverse 4.
21:19 Thalia_UM
4.7
21:21 pdurbin
ok
21:28 pdurbin
It's getting dark out and it's time to pick up the kids. Bye!
21:28 pdurbin left #dataverse
21:34 axfelix joined #dataverse
22:59 axfelix joined #dataverse
23:22 axfelix joined #dataverse
