Time
S
Nick
Message
01:16 axfelix joined #dataverse
01:19 axfelix joined #dataverse
03:04 axfelix joined #dataverse
06:31 jri joined #dataverse
08:09 jri joined #dataverse
08:11 jri joined #dataverse
08:12 jri joined #dataverse
10:28 donsizemore joined #dataverse
11:13 andrewSC joined #dataverse
12:04 donsizemore joined #dataverse
14:44 pameyer joined #dataverse
14:50 ferrys joined #dataverse
14:53 ferrys
Hi all! I am collaborating with a team and they need help going through the process of creating a dataset and uploading a data file programmatically through python. Does anyone have experience with this? I'm having trouble determining the best way to get the dataset id for the dataset I've just created
14:56 pameyer
hi ferrys: not sure if this is the recommended way or not, but I'd use the python requests library and native API
14:57 pameyer
the JSON from the create dataset API has the dataset id in it
15:00 ferrys
Could you point me towards that?
15:00 ferrys
@pameyer that was what I was trying to do by following the snippet here http://guides.dataverse.org/en/latest/api/native-api.html
15:02 pameyer
curl -s -H "X-Dataverse-key: $k_d" -X POST --upload-file $fn $h/api/dataverses/$dv_d/datasets
15:02 pameyer
dset_id=`echo $r | jq .data.id`
15:03 pameyer
curl -s -H "X-Dataverse-key: $k_d" -X POST -F "file=@${dfile}" $h/api/datasets/${dset_id}/add
15:03 pameyer
^ cut&paste from one of my scripts
15:03 pameyer
I'd use the requests library differently than that snipped
15:04 pameyer
where's it failing for you / your collaborators?
15:06 pameyer
I'd guess here: `r = requests.post(url_persistent_id, data=payload, files=files)`
15:28 ferrys joined #dataverse
15:29 pdurbin
ferrys: I saw you got disconnected. Did you see what pameyer was saying?
15:29 ferrys
@pdurbin I checked the logs, I saw!
15:30 pdurbin
ferrys: cool. Also there's something called pyverse: https://github.com/IQSS/dataverse-client-python/issues/43#issuecomment-327180842
15:31 ferrys
pameyer: its failing because I don't know how to get the dataset id of the dataset I just created within the python file
15:32 pameyer
pdurbin: it might not be a bad idea to take the `key` param out of the url in that snippet from the docs
15:32 pdurbin
To back up, we're basically looking for a maintainer for that dataverse-client-python code. It is or was used in production by https://osf.io but it was written in the DVN 3 days and doesn't use all the new Dataverse 4 APIs.
15:33 ferrys
pameyer: do you think it's even possible to create a dataset and upload files to it in the same python script?
15:34 pameyer
ferrys: it's definitely possible
15:34 pameyer
I have a single bash script that does this (mainly for testing)
15:35 ferrys
Would you mind emailing it to me? ferrys at bu.edu
15:35 ferrys
pameyer: i may be able to work it into python
15:36 pameyer
ferrys: no problem; it should be a pretty straightforward translation to python requests
15:36 ferrys
pameyer: thanks!
15:39 pdurbin
pameyer: good idea. I just created this: API Guide: Favor passing API token in header rather than as query parameter "key" · Issue #4254 · IQSS/dataverse - https://github.com/IQSS/dataverse/issues/4254
15:39 pameyer
no problem
15:39 pameyer
pdurbin: cool - I'd though there might have already been one for it
15:39 pdurbin
we can close it as a dup if necessary :)
15:40 pdurbin
ferrys: if you're happy with curl examples, you can find both a SWORD and native API example of creating a dataset in this script that gets executed every time we run the API test suite on the phoenix server: https://github.com/IQSS/dataverse/blob/v4.8.1/scripts/search/tests/create-all-and-test#L8
15:44 pdurbin
andrewSC isn't here at the moment but might have time after April to help with dataverse-python-client: http://irclog.iq.harvard.edu/dataverse/2017-08-16#i_55811
15:45 andrewSC joined #dataverse
15:51 Guest80744 joined #dataverse
15:59 ferrys
pameyer: pdurbin pointed out that the dataset id is returned in the python response object! so hopefully that does the trick
16:00 pdurbin
well, in the response, yeah
16:00 pdurbin
SWORD returns the DOI
16:00 pdurbin
native retuns the database id of the dataset
16:00 pdurbin
on create
16:02 pdurbin
Once https://github.com/IQSS/dataverse/commit/272b9bf gets merged the native API will also return the DOI.
16:02 pdurbin
andrewSC: oh hey, I was just talking about you. ^^ :)
16:02 pdurbin
all good stuff
16:03 andrewSC
:D
16:05 pdurbin
andrewSC: I added you to my "contributors" spreadsheet, but can I also add you to this newer "dev efforts by the community" spreadsheet? https://docs.google.com/spreadsheets/d/1pl9U0_CtWQ3oz6ZllvSHeyB0EG1M_vZEC_aZ7hREnhE/edit?usp=sharing
16:05 andrewSC
would this be for the python lib?
16:05 pdurbin
yeah
16:05 andrewSC
err client i should say
16:06 andrewSC
do you have a timeline or date you need it completed by/do you forsee any major api changes on the roadmap soon?
16:08 pdurbin
I don't foresee any major API changes. There is no deadline. This spreadsheet is meant to gather the community dev efforts together. To provide a point of contact. For more on this spreadsheet, please see https://groups.google.com/d/msg/dataverse-community/brxCn1E9tX0/VbsNz4u8BgAJ
16:09 pdurbin
andrewSC: the "Project" and "Status" can be whatever you want. You can even request edit access to the spreadsheet if you want.
16:11 andrewSC
:)
16:11 andrewSC
Sure you can put me down
16:20 pdurbin
andrewSC: added! Thanks! Does what I wrote look ok?
16:30 andrewSC
pdurbin: looks good :)
16:34 pdurbin
great
16:55 donsizemore joined #dataverse
18:13 jri joined #dataverse
18:25 ferrys joined #dataverse
20:09 andrewSC joined #dataverse
20:29 axfelix joined #dataverse
20:44 axfelix joined #dataverse
21:07 axfelix joined #dataverse
21:41 axfelix joined #dataverse
22:07 axfelix joined #dataverse
22:30 axfelix joined #dataverse
22:31 axfelix joined #dataverse
