IRC log for #dataverse, 2017-08-15

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.

All times shown according to UTC.

Time	Nick	Message
05:15		axfelix joined #dataverse
11:31		donsizemore joined #dataverse
11:39		donsizemore joined #dataverse
14:31		pameyer joined #dataverse
15:07		jhand joined #dataverse
15:21		donsizemore joined #dataverse
17:28	andrewSC	you guys know where I should look for custom metadata fields?
17:29	andrewSC	Basically I'm trying to create a dataset via api but i'm failing validation on some custom metadata fields i've set to be required for the creation of a dataset
17:30	andrewSC	I've checked to see what the keys might be by querying a dataset in solr and seeing what the keys are but they don't work
17:30	andrewSC	don't work when i try to use the same keys to create the dataset*
17:30	pameyer	have you tried creating one through the UI, and calling the dataset API to see what it looks like?
17:31	andrewSC	mmmmmmm I have not lemme try that
17:31	pameyer	that's usually my first approach to create api problems
17:40	pdurbin	andrewSC: you're getting Solr errors having to do with your custom metadata fields?
17:44	pdurbin	pameyer: good idea to try the UI
17:44	pdurbin	some day we should work on this "Dataverse project take ownership of documentation for creating metadata blocks" issue: https://github.com/IQSS/dataverse/issues/3168
17:47	andrewSC	pdurbin: i don't think so? I'm seeing a validation failed error from "edu.harvard.iq.dataverse.api.datadeposit.CollectionDepositManagerImpl" in the logs
17:48	andrewSC	funny enough I just use the /api/datasets endpoint to pull in a dataset and sure enough the fields it's failing on are identical to the ones i tried, casing and all :/
17:48	andrewSC	i know the tool creating the datasets itself is fine because i tested it against the demo dataverse you guys host and that went without a hitch
17:49	pdurbin	CollectionDepositManagerImpl is how SWORD calls the CreateDatasetCommand
17:50	andrewSC	mmmmmmmmmm
17:50	pdurbin	andrewSC: if you pastebin more of the stacktrace I'd be happy to take a look
17:50	pdurbin	or email it to supportdataverse.org
17:57	andrewSC	pdurbin: https://gist.github.com/andrewSC/16891c30712b444d45799d3a9b9f1d2d
17:57	andrewSC	just looked through this
17:58	andrewSC	there's some stuff in there particularly the subject exists already message i'm not sure what to make of
17:58	pdurbin	ah, you've bumped up logging. good
17:58	andrewSC	:)
17:59	andrewSC	here's the python code snippet: https://gist.github.com/andrewSC/d7fb41256e2011cf1094550090bc29d8
17:59	andrewSC	lines 8-10 are something I had to write since the lib actually uses the href from the dataverse rather than the one specified by the connection
18:01	pdurbin	I'm used to seeing dcterms:title but there's weird stuff like dcterms:LASitemType in your XML. Maybe a red herring.
18:01	andrewSC	yeah the LASitemType is a metadata field we added as required to create a dataset
18:02	andrewSC	there's a handful of custom fields we have added as required but idk where those changes were made/how to add custom fields?
18:03	pdurbin	And you plan to create datasets with that custom field using SWORD (XML) rather than the native API (JSON)?
18:04	andrewSC	If the python lib uses SWORD then I'd say we're trying to create datasets that way
18:04	andrewSC	looking at that fine log again it seems like those failed validations are because the code never got to them in the first place? since it exists early from a duplicate subject field?
18:06	pdurbin	The "Validation failed" message is coming from CreateDatasetCommand.
18:07	pdurbin	andrewSC: want to see if you're seeing the same trouble with the native api?
18:07	andrewSC	yeah lemme give it a shot
18:08	pdurbin	Cool. pameyer has used the native api with his own custom metadata blocks so it should work. I'm not sure if he's using python or curl.
18:10	pdurbin	andrewSC: does it work fine to create a dataset in the gui?
18:14	andrewSC	mhmm
18:17	pdurbin	good
18:20	andrewSC	pdurbin: native api confirmed working
18:21	andrewSC	how funny
18:23	pdurbin	andrewSC: I'm not super surprised to be honest. For SWORD you have to populate a database table
18:24	andrewSC	mmmmmmmmm lol
18:24		donsizemore joined #dataverse
18:25	pdurbin	andrewSC: I mean, it might be possible. I've just never tried it.
18:26	pdurbin	using a custom metadata block with a required field with sword, i mean
18:27	andrewSC	mmmmmmm
18:28	pdurbin	andrewSC: are you ok with using the native api?
18:29	andrewSC	mhmm I just wanted to try and use the python dataverse module rather than rolling my own
18:29	andrewSC	i mean that part of the code is factored out well enough where it'll be an isolated change to use the native api with python requests or w/e
18:30	andrewSC	unless there's a dataverse python module that already leverages the native pi
18:30	andrewSC	api*
18:31	pdurbin	andrewSC: we would love a pull request for https://github.com/IQSS/dataverse-client-python that adds more native api support if you'd like to help. :)
18:32	andrewSC	:)))
18:34	andrewSC	I want to believe I have the time to contribute
18:35	andrewSC	would be solid to be able to point to contributions to OSS outside of maintaining packages for a distro i like
18:37	pdurbin	andrewSC: well, maybe you can open source whatever you end up with by using requests
18:38	andrewSC	mhmm
18:40	andrewSC	the rub is I've been using a grad student to write the tool so I can focus on other tasking (I was given the opportunity to own infrastructure for the lab) and I worry his code quality isn't going to be up to par for OSS
18:40	andrewSC	he gets by on the tooling with some PRs from me for bugfixes
18:40	andrewSC	but I wouldn't consider his code OSS ready
18:43	pdurbin	I don't consider my code OSS ready. :)
18:44	andrewSC	haha
18:57	pameyer	I've been using curl for the most part
19:01		bsilverstein joined #dataverse
19:38	pdurbin	pameyer: I'm switching your stuff to getDescriptionPlainText
19:40	pameyer	cool
19:44	pdurbin	pameyer: https://github.com/IQSS/dataverse/commit/47feb73

Connect via chat.dataverse.org to discuss Dataverse (dataverse.org, an open source web application for sharing, citing, analyzing, and preserving research data) with users and developers.